''device for obtaining a focused image of a specimen, method, computer readable medium, devi

专利摘要:
device and method for obtaining a focused image of a specimen, method for obtaining an image of a specimen and computer readable medium this application relates to the field of imaging and more particularly to systems and techniques for obtaining and capturing images . the device for obtaining a focused image of a specimen comprises an objective lens (120, 1410, 1552, 1720) arranged for examination of the specimen (1501'); a slow focusing stage (140) coupled to the objective lens (120, 1410, 1552, 1720), wherein the slow focusing stage (140) controls movement of the objective lens (120, 1410, 1552, 1720); an oscillator focus stage (150) including an oscillator lens, wherein the oscillator focus stage (150) moves the oscillator lens; a focus sensor (160) that provides focus information in accordance with light transmitted through the oscillator lens; at least one electrical component that uses the focus information to determine a metric and a first objective lens focus position (120, 1410, 1552, 1720) according to the metric, wherein at least one electrical component sends position information the slow focusing stage (140) to move the objective lens (120, 1410, 1552, 1720) to the first focus position; and an image sensor (110, 1536, 1710) that captures an image (110') of the specimen (1501') after the objective lens (120, 1410, 1552, 1720) is moved to the first focus position.
公开号:BR112012009241A2
申请号:R112012009241-3
申请日:2010-10-18
公开日:2020-09-01
发明作者:Gregory C.Loney；Bikash Sabata；Chris Todd；Glenn Stark
申请人:Ventana Medical Systems,Inc.；
IPC主号:

专利说明:

m
¥ "DEVICE AND METHOD FOR GETTING A FOCUSED IMAGE OF A SPECIMEN, METHOD FOR GETTING AN IMAGE OF ONE AND HALF COMPUTER-READable SPECIMEN IMAGE"
Field of Invention
5 This application relates to the field of imaging and more particularly to systems and techniques for obtaining and capturing images.
Background of the Invention The identification of changes in cellular structures indicative of disease remains a molecular imaging key to the best
10 understanding .in medicinal science.
Microscopy applications are applicable to microbiology (e.g. Gram stain, etc.), plant tissue culture, animal cell culture (e.g. phase contrast microscopy, etc.), molecular biology, immunology (e.g. e.g. ELISA, etc.), cell biology (e.g. immunofluorescence, analysis of
15 chromosome, etc.), confocal microscopy, live and time-lapse cell imaging, three-dimensional and serial imaging.
There have been advances in confocal microscopy that have unlocked many of the secrets that occur within the cell and changes at the transcriptional and translational level can be detected by the use of markers of
20 fluorescence.
The advantage of the confocal approach stems from the ability to _ _generate. image individual optical sections at high resolution in sequence through the specimen.
However, there still remains a need for systems and methods for the digital processing of pathological tissue images that provide accurate analysis of pathological tissue, at a cost.
25 relatively low.
It is a desirable goal in digital pathology to obtain high resolution digital images for visualization in a short period of time.
Current manual methods so the pathologist views the slide through the lens
%, 0 high resolution for viewing in a short time. Current manual methods so that the pathologist views the slide through the ocular lens of a microscope allow a diagnosis by inspection of cell characteristics or the count of stained versus unstained cells. Automated methods are desirable so that digital images are collected, viewed on high resolution monitors and can be shared and archived for later use. It is advantageous that the scanning process is carried out efficiently with a high throughput and with high resolution and high quality images.
10 In conventional virtual microscopy systems, imaging techniques can produce individual images that may be significantly out of focus over a large portion of the image. Conventional imaging systems are constrained to a single focal length for each individual snapshot taken by the camera, so each of these 15 "fields of view" have areas that are out of focus when the specimen in question being scanned does not have a uniform surface. At the high magnification levels employed in virtual microscopy, specimens with a uniform surface are extremely rare. Conventional systems use a pre-focusing technique 20 to address the high proportion of out-of-focus images that is based on a two-step process that includes: 1) determining, in a first step, the best focus in a matrix of points, separated by "n" image frames, arranged in a two-dimensional grid extended on top of a pathological section; and 2) in another step, move to each focus point and acquire an image frame. For points between these best focus points, focus is interpolated. While this two-step process can reduce or even eliminate out-of-focus images, the process results in a significant loss in the speed of acquiring side-by-side images.
the -Cl-.
w Consequently, it would be desirable to provide a system that overcomes the significant problems inherent in conventional imaging systems and efficiently delivers high-quality, focused images at a high throughput.
5 DESCRIPTION OF THE INVENTION According to the system described herein, a device for obtaining a focused image of a specimen includes objective lenses arranged for the examination of the specimen. A slow focusing stage is attached to the objective lenses, and the slow focusing stage controls the movement of the 10 objective lenses. An oscillator focus stage includes oscillator lenses, and the oscillator focus stage moves the oscillator lenses. A focus sensor provides focus information according to the light transmitted through the oscillator lens. At least one electrical component uses the focus information to determine a metric and a first focus position of the 15 objective lenses according to the metric, where the electrical component sends the position information to the slow focusing stage to move the objective lenses to the first focus position. An image sensor captures an image of the specimen after the objective lens is moved to the first focus position. An XY drive stage can be included, the specimen is arranged on the XY drive stage, and in which the electrical component controls the movement of the XY drive stage. The movement of the XY moving stage can be phase-closed with the movement of the oscillator lens. The oscillator focus stage may include a voice coil driven bent assembly that moves the oscillator lenses in a translational motion. Oscillator lenses can be moved at a resonant frequency that is at least 60 Hz, and where the electrical component uses the focus information to perform at least 60 focus calculations per second. The focus sensor and oscillator focus stage can be rh adjusted to operate in a bi-directional manner, whereby the focus sensor outputs focus information on both an upper and lower portion of a sinusoidal waveform of the motion of the hands. oscillator lenses at the resonant frequency.
The metric can include contrast information,
5 sharpness information, and/or chroma information.
The focus information can include information for a plurality of zones of a focus window that is used during the specimen focus scan.
An electrical component may control movement of the XY drive plate, and wherein information from at least a portion of the plurality of zones is used in the
10 determining the speed of the XY drive plate.
A focus sensor field of view can be rotated relative to the image sensor field of view.
In accordance with the system further described in the present invention, a method for obtaining a focused image of a specimen is
15 provided.
The method includes controlling the movement of objective lenses arranged for specimen examination.
Oscillator lens movement is controlled and focus information is provided according to light transmitted through the oscillator lens.
The focus information is used to determine the metric and determine the first focus position of the lenses
20 objectives according to the metric. Position information is sent that is used to move the objective lens to the first focus position.
A first focus position can be determined as the best focus position,
and the method may further include capturing an image of the specimen after the objective lens is moved to the best focusing position.
At
25 oscillator lenses can be moved at a resonant frequency that is at least 60 Hz, and at least 60 focus calculations can be performed per second.
The metric can include sharpness information, contrast information, and/or chroma information.
Focus information can include
W and information for a plurality of zones of a focus window that is used during a specimen focus scan. The movement of an XY drive stage on which the specimen is disposed can be controlled, and information from at least a portion of the plurality of zones can be used in determining a speed of the XY drive stage. The movement of the XY drive stage can be controlled to provide forward and backward translational scanning of the specimen.
In accordance with the system further described in the present invention, a method for imaging a specimen includes establishing a nominal focus plane. The specimen is positioned at a start position that has x and y coordinates associated with it. First, processing is performed in a single pass over said specimen. The first processing includes determining, for each of said plurality of points, a focus position by using oscillator lenses, and acquiring, for each of said plurality of points, a frame according to said focus position. In accordance further with the system described in the present invention, a computer readable medium comprising code stored therein for obtaining a focused image of a specimen in accordance with any of the steps mentioned above. Furthermore, a computer-readable medium may comprise the code stored therein to execute any of the more of the processes described below. In accordance with the system further described in the present invention, a device for a microscope stage includes a moving stage block and a base block which guides the moving stage block. The base block includes a first block that is substantially flat and a second block that is triangular in shape, wherein the first block and the second block guide the platinum block to move in a translational direction.
The first block and the second block can be supported on raised ledges on a base plate.
The first block and the second block can be made of glass.
A plurality of button elements can be arranged on the moving platen block
5 which contacts the first block and the second block, and the button elements can allow movement of the moving platen block only in the translational direction.
Button elements can be spherical in shape and made of thermoplastic.
At least two of the plurality of button elements may be arranged to face each other on each
10 side of the triangular shape of the second block, and wherein at least one button of the plurality of button elements contacts the first block on a flat face thereof.
The positions of the plurality of button elements on the moving platen block can form a triangle.
Each of the plurality of button elements can support an equal weight during
15 platinum movement.
The moving platinum block may be formed to have a center of gravity at a centroid of the triangle formed by the positions of the plurality of button elements.
A cantilever arm assembly can be provided and a bent member can be provided that has a first end rigidly coupled to the arm assembly.
20 cantilever and a second end coupled to a locating center of mass on the moving platinum block.
The cantilever arm assembly may include a cantilever arm coupled to a bearing block that passes through a recirculating bearing design on a rail.
Driving the housing block on the rail can cause the bent element to apply
25 a force on the platinum drive block.
Bending stiffness of the bent element can isolate the platinum block from moving from up and down movements of the cantilever arm assembly.
The base block can form another drive plate in one direction W M
E 7 4 *
D —. .-— — perpendicular to the translational direction of the moving platinum block. Repeatability in motion can be provided in the range of 150 nanometers. The repeatability in motion can be orthogonal to the motion page and the base block translational directions.
In accordance with the system further described in the present invention, a blade storage device includes a shelf, a buffer, a blade handle that moves a first blade between the shelf and the storage, and an XY stage. The XY stage moves a second blade in connection with a sweep of the second blade, and at least one Blade handle function corresponding to the first blade is performed in parallel with at least one XY stage function corresponding to the second blade. The blade handle can move the first blade and the second blade between the shelf, the storage and the XY stage and can move with at least three degrees of freedom. The XY stage may include a blade receiving head that moves the blades from the stocker to the XY stage. An imaging device may generate an image of the first blade and the second blade, and may include a focusing system and a camera. The targeting system may include a dynamic targeting system. The Blade knob function performed in parallel with the XY piatina function can provide a time gain of at least 10°/o. The blade handle may include a blade receiving head that includes a mechanical pickup device and/or a vacuum pickup device. The store may include a plurality of store positions that accept a plurality of 25 blades. At least one spool position of the spool may be a position used to capture a thumbnail image of a slide. The shelf may include at least a main tray and a bypass tray, and a slide arranged in the bypass tray is processed prior to
W b any blade arranged in the main tray.
In accordance with the system further described in the present invention, a method for blade storage includes providing a shelf and storage.
A first blade is moved between a
5 silverware and the store.
A second blade is moved in or out of the store in connection with a sweep of the second blade.
The movement of the first blade between the shelf and the store can be performed in parallel with the sweep of the second blade.
The second blade sweep may include a focusing operation and a hovering operation.
10 image capture.
Moving the first blade in parallel with the sweep of the second blade can provide a time gain of at least 10%. The second blade sweep may include a dynamic focusing operation.
The store may include a plurality of store positions which includes at least: a store position of
15 camera and a return storage position.
The method may further include capturing a thumbnail image of the first blade and/or second blade when the first blade and/or second blade is in the camera store position.
In addition to the system described in the present
In the invention, a sheet storage device includes a first shelf, a second shelf, a first XY stage and a second XY stage.
The first XY stage moves a first blade into or out of the first shelf in connection with a sweep of the first blade.
The second XY stage moves a second blade in or out of the second
25 shelf in connection with a sweep of the second blade.
At least one function of the first stage XY that corresponds to the first blade is performed in parallel with at least one function of the second stage XY that corresponds to the second blade.
The first and second shelf
- 6 shelf can form parts of a single shelf. An imaging device can generate an image of the first blade and the second blade. The first XY stage and the second XY stage may each include a blade receiving head.
In accordance with the system further described in the present invention, a blade sweeping device includes a rotatable tray and at least one recess arranged in the rotatable tray. The recess is sized to receive a blade, and the recess stabilizes the blade in a sweeping position as a result of rotation of the rotatable tray. The recess may include a plurality of protuberances which stabilize the blade and may include a plurality of recesses arranged in a circumferential ring of the rotatable tray. An imaging system may be included, and at least one component of the imaging system moves in a radial direction of the rotatable tray 15. The imaging system component can move incrementally in the radial direction which corresponds to one full rotation of the rotating tray. The recess may be sized to receive a blade having a length that is greater than a blade width, and the blade length may be oriented in a radial direction of the rotatable tray. The recess may be sized to receive a blade having a length that is greater than a blade width, and the blade width may be oriented in a radial direction of the rotatable tray. In accordance with the system further described in the present invention, a method of sweeping a blade includes arranging the blade in at least one recess of a rotatable tray and rotating the rotatable tray. The recess is sized to receive a Blade, and the recess stabilizes the blade in a sweeping position as a result. W
F 0 of rotation of the rotating tray. The recess may include a plurality of protrusions which stabilize the blade and may include a plurality of recesses disposed in a circumferential ring of the rotatable tray. The method may further include providing an imaging system and moving at least one component of the imaging system in a radial direction of the rotatable tray. The imaging system component can be moved incrementally in the radial direction which corresponds to one full rotation of the rotating tray. The recess may be sized to receive a blade having a length that is greater than a blade width, and wherein the blade length is oriented in a radial direction of the rotatable tray. The recess may be sized to receive a blade having a length that is greater than a blade width, and wherein the blade width is oriented in a radial direction of the rotatable tray.
15 BRIEF DESCRIPTION OF THE DRAWINGS The embodiments of the system described herein will be explained in more detail herein on the basis of the Figures of the drawings, which are briefly described as follows. Figure 1 is a schematic illustration of an imaging system of a scanning microscope and/or other scanning device which may include various component devices used in connection with scanning and imaging a digital pathology sample in accordance with with various realizations of the system described in this. Figure 2 is a schematic illustration showing an imaging device that includes a focusing system in accordance with an embodiment of the system described herein. Figures 3A and 3B are schematic illustrations of an embodiment of the control system showing that the control system can
- 11
K b include appropriate electronic components.
Figure 4 is a schematic illustration showing the oscillator focus stage in more detail according to an embodiment of the system described herein.
5 Figures 5A to 5E are schematic illustrations showing an interaction focusing operations according to the system described herein. Figure 6A is a schematic illustration of a plot showing the lead waveform of optical sharpening determinations 10 of the oscillator focus in accordance with an embodiment of the system described herein. Figure 6B is a schematic illustration showing a plot of calculated sharpening values (Zs) for a portion of the sine wave motion of the oscillator lenses. Figures 7A and 7B are schematic illustrations showing determinations and focusing adjustments of a (pathological) specimen according to an embodiment of the system described herein. The figure. 8 is a schematic illustration showing an example of a sharpening profile that includes a sharpening curve and contrast ratio for each sharpening response at multiple points that are sampled by the oscillator focusing optics in accordance with an embodiment of the described system. in this. Figure 9 shows a functional control loop block diagram illustrating the use of the contrast function to produce a control signal to control the slow focus stage.
Figure 10 is a schematic illustration showing the focus window that is broken into zones in connection with focus processing according to an embodiment of the system described herein. Figure 11 shows a graphical illustration of sharpening values
'm— &
Different Y that can be obtained at points in time in an achievement according to techniques in this. Figure 12 is a flowchart showing real-time focus processing while scanning a specimen under examination in accordance with an embodiment of the system described herein. Figure 13 is a flowchart showing slow-focus stage processing in accordance with an embodiment of the system described herein. Figure 14 is a flowchart showing image capture processing in accordance with an embodiment of the system described herein.
Figure 15 is a schematic illustration showing an alternative arrangement for focus processing in accordance with an embodiment of the system described herein. Figure 16 is a schematic illustration showing an alternative arrangement for focus processing according to another embodiment of the system described herein. Figure 17 is a flowchart showing the processing to acquire a pathology mosaic image on a slide according to an embodiment of the system described herein. Figure 18 is a schematic illustration showing an implantation of a precision stage (e.g., a platinum -_Y portion) of an XY stage in accordance with an embodiment of the system described herein. Figures 19A and 19B are more detailed views of the platinum block for moving the precision plate according to an embodiment of the system described herein.
Figure 20 shows an implantation of a total XY composite stage in accordance with the precision stage features discussed herein and which include a Y stage, an X stage and a baseplate in accordance with an embodiment of the system described herein.
+
P 13
W + -- & Figure 21 is a schematic illustration showing a blade storage device according to an embodiment of the system described herein. The figure. 22A is a flow chart showing blade storage processing according to an embodiment of the system described herein in connection with a first blade. Figure 22B is a flowchart showing blade storage processing in accordance with an embodiment of the system described herein in connection with a second blade.
Figures 23A and 23B show timing diagrams using blade storage techniques in accordance with embodiments of the system described herein and illustrating the time savings in accordance with various embodiments of the system described herein. Figure 24 is a schematic illustration showing a blade storage device according to another embodiment of the system described herein. Figure 25A is a flowchart showing slide storage processing in connection with a first slide according to a described embodiment of the system for a slide storage device having two XY composite stages for slide processing. Figure 25B is a flowchart showing blade storage processing in connection with a second blade according to a described system embodiment for blade storage device 25 having two XY composite stages for blade processing. Figure 26 is a schematic illustration showing a blade storage device according to another embodiment of L_
0·
P "T 14
W « « system described in this. Figure 27 is a schematic illustration showing another view of the blade storage device according to the Figure. 26. Figures 28A to 28J are schematic illustrations showing 5 blade storage operations of the blade storage device of Figures 26 and 27 according to an embodiment of the system described herein. Figure 29 is a schematic illustration showing a lighting system for illuminating a blade using a light emitting diode (LED) lighting assembly 10 in accordance with an embodiment of the system described herein. Figure 30 is a schematic illustration showing a more detailed view of an embodiment for an LED lighting assembly according to the system described herein.
Figure 31 is a schematic showing an exploded view of a specified deployment of an LED lighting assembly in accordance with an embodiment of the system described herein. Figure 32 is a schematic illustration showing a high speed blade scanning device in accordance with an embodiment of the system described herein which can be used in connection with a digital pathology image.
Figure 33 is a schematic illustration showing a recess in a tray of the high speed blade scanner in more detail according to an embodiment of the system described herein.
25 Figure. 34 is a schematic illustration showing an imaging trajectory starting at a first radial position with respect to the blade for imaging with a specimen on the blade in the recess.
© 0 0 Figures 35A and 35B are schematic illustrations showing an alternative arrangement of blades on a rotating blade holder according to another embodiment of the system described herein. Figure 36 is a schematic illustration showing an imaging system 5 in accordance with an embodiment of the system described herein which includes an objective arranged to examine a specimen on a slide. Figure 37 is a flowchart showing slide scanning high speed by using a rotatable tray in accordance with an embodiment of the system described herein. Figure 38 is a schematic illustration showing an optical duplication imaging system in accordance with an embodiment of the system described herein.
Figures 39A and 39B are schematic illustrations of the optical doubling image system 15 showing the back and forth movement of the first tube lenses and the second tube lenses in front of the image sensor according to an embodiment of the system described herein. .
DESCRIPTION OF EMBODIMENTS OF THE INVENTION Figure 1 is a schematic illustration of an imaging system 5 of a scanning microscope and/or other scanning device which may include various component devices used in connection with scanning and scanning. pathology sample imaging according to the various embodiments of the system described herein. The imaging system 5 may include an imaging device with focusing system 10, a blade plating system 20, a blade storage system 30 and a lighting system 40, among other component systems 50 , as discussed further in detail elsewhere in this document. It is also worthy of attention b W —
W that the system described herein can be used in connection with microscope slide scanning instrument architectures and techniques for image capture, splicing and magnification as described in Patent Application Publication No. US 2008/0240613 Al a Dietz et al ., entitled "Digital Microscope 5 S/ide Scanning System and Methods", which is incorporated herein by reference, which includes features in connection with reconstituting an image at a magnification without substantial loss of accuracy and displaying or storing the reconstituted image .
Figure 2 is a schematic illustration showing an imaging device 100 of an optical scanning microscope and/or other appropriate imaging systems that include components of a focusing system to produce focused images of a pathological sample 101 and/or other objects arranged on a sheet in accordance with an embodiment of the system described herein. The focusing system described in this provides for determining the best focus for each snapshot as a snapshot is captured, which can be referred to as "real-time focusing." The devices and techniques provided in this lead to significant reductions in the time required to form a digital image of an area and a pathology slide. The system described herein integrates the steps of the two-step approach of conventional systems and essentially eliminates the time required for pre-focusing. The system described in this provides for the creation of a digital image of a specimen on a microscope slide by using real-time processing to capture snapshots where the total time to capture all 25 snapshots is less than the time required by one method. by using a focus point predetermination step for each snapshot prior to capturing the snapshots.
Imaging device 100 may include an imaging sensor 110, such as a charge-coupled device (CCD) and/or complementary metal oxide semiconductor (CMOS) image sensor, which may be part of a camera 111 that captures digital images of pathology.
110 imaging sensor can receive transmitted light
5 of a microscope objective 120 transmitted through tube lenses 112, a beam splitter 114 and which includes other components of a light transmitting microscope such as a condenser 116 and a light source 118 and/or other appropriate optical components 119 The 120 microscope objective can be infinitely corrected.
In one embodiment, the beam splitter 114
10 can provide to distribute approximately 70% of the light beam source directed towards the image sensor 110 and the remaining portion of approximately 30°/o directed along a path to the oscillator's focusing stage 150 and the focus sensor. 160. The tissue sample 101 from which an image is generated can be arranged on a plate of
15 XY movement 130 which can be moved in the X and Y directions and which can be controlled as discussed further elsewhere in this document.
A slow focusing stage 140 can control the movement of the microscope objective 120 in the Z direction to focus an image of the pathological 101 that is captured by the image sensor 110.
Slow focusing 140 may include a motor and/or other devices suitable for moving the microscope objective 120. An oscillator focusing stage 150 and a focus sensor 160 are used to provide fine focusing control for real-time focusing. according to the system described in this.
In various embodiments, the focus sensor 160 can be a
25 CCD and/or CMOS sensor.
Oscillator 150 focusing stage and 160 focus sensor provide real-time focusing according to sharpness values and/or other metrics that are quickly calculated during the imaging process to achieve better focus for each snapshot image as it is captured.
As discussed in further detail elsewhere in this document, the oscillator focusing plate
150 can be moved on a frequency, for example in one move
5 sinusoidal, which is independent and exceeds the motion frequency practicable for the slower motion of the microscope objective 120. Multiple measurements are performed by the focus sensor 160 of focus information to visualize the pathological through a range of motion of the oscillator focusing 150. The focusing electronics and control system 170 may include electronics for controlling the focus sensor and the oscillating focus stage 150, a master clock, the electronics for controlling the slow-focus stage 140 ( Z direction), XY 130 drive plate, and other components of a realization of a system in accordance with techniques in this.
The focus electronics and control system 170 can be used to perform sharpness calculations using information from the focusing stage from oscillator 150 and focus sensor 160. Sharpness values can be calculated across at least a portion of a sinusoidal curve defined by the oscillator motion.
The focusing electronics and control system 170 can then use the information to determine the position for the best pathological image focus and command the slow focus stage 140 to
- -move the microscope objective 120 to a desired position (along the Z axis as shown) to obtain the best image focus during the imaging process.
The control system 170 may also use the information to control the speed of the XY drive stage 120, for example the speed of motion of the stage 130 in the Y direction.
In one embodiment, sharpness values can be computed by differentiating the contrast values of neighboring pixels, squaring them and > and and summing the values to form a score. Various algorithms for determining sharpening values are discussed further elsewhere in this document.
In various embodiments In accordance with the system described herein, and in accordance with components discussed elsewhere herein, a device for creating a digital image of a specimen on a microscope slide includes: a microscope objective that is infinitely corrected ; a beam splitter; camera focusing lenses; a high resolution camera; a group of sensor focus lenses; an oscillator focusing page 10; a focusing sensor; a coarse focusing (slow) stage; and focus electronics. The device can allow to focus the lens and capture each instant through the camera without needing to pre-determine the focus point for all the snapshots prior to taking the snapshot, and wherein the total time to capture 15 all snapshots is less than the time required per system that requires a step of pre-determining the focus points for each snapshot prior to capturing the snapshot. The system may include computer controls for: i) determining a first focus point on the pathologic to establish a nominal focus plane by moving the coarse focus stage 20 through the entire z-range and monitoring the sharpness values; ii) positioning the pathological in x and y to start in a corner of an area of interest; iii) setting the oscillator fine-focus stage to move, wherein the oscillator focus stage is synchronized with a master clock that also controls the speed of the xy stage; iv) commanding the stage to move from one frame to the adjacent frame, and/or v) producing a trigger signal to acquire a frame on the image sensor and trigger a light source to create a light pulse. Furthermore, according to another embodiment, the system described in
· no.
C —-""16 m 20
The present document may provide a computer-implanted method of creating a digital image of a specimen on a microscope slide.
The method may include determining a scan area that comprises a region of the microscope slide that includes at least a portion of the specimen. The scan area can be divided into a plurality of j snapshots. Snapshots can be captured using a microscope objective and camera, whereby focusing the objective and microscope and capturing each snapshot through the camera can be conducted for each snapshot without the need to predetermine a point of view. focus for all snapshots before capturing the snapshots. The total time to capture all snapshots may be less than the time required by a method that needs a step to predetermine the focus points for each snapshot before capturing the snapshots.
Figure 3A is a schematic illustration of an embodiment of the focus electronics and control system 170 including the focus electronics 161, a master clock 163, and the platinum control electronics 165. Figure 3B is a schematic illustration. of one embodiment of focusing electronics 161. In the illustrated embodiment, focusing electronics 161 may include suitable electronics such as a suitable fast ND converter 171 and a field programmable gate array (FPGA) 172 with microprocessor 173 which can be used to make sharpening calculations. A/D converter 171 can receive information from focus sensor 160 25 which is coupled to FPGA 172 and microprocessor 173 and used to output sharpening information. The master clock included in the 170 can provide the master clock signal to the focus electronics 161, the platinum control electronics 165, and other system components. Hi B
B 21 4 P
The electronic platinum control component 165 can generate control signals used to control the slow focus stage 140, XY motion stage 130, oscillator focus stage 150, and/or other control signals and information, as discussed elsewhere. place in this document. The 5 FPGA 172 can provide a clock signal to the focus sensor 160, among other information. Measurements in the Laboratory show a calculation of sharpening in a frame of 640 x 32 pixels can be done in 18 microseconds, easily fast enough for proper operation of the system described in this document. In one embodiment, focus sensor 160 may include a windowed monochrome CCD camera 10 for 640 x 32 range, as discussed elsewhere herein. The scanning microscope may acquire a 1D or 2D array of pixels including contrast information, and/or intensity information in RGB or some other color space, as discussed elsewhere in this document. The system finds the best focus points across a large field, for example on a 25mm x 50mm glass slide. Many commercial systems sample the scene produced by a 20x, 0.75 NA microscope objective with a CCD array. Given the objective and condenser NA of 0.75 and a wavelength of 500 nm, the lateral resolution of the optical system is about 0.5 micron. To sample this resolution element at the Nyquist frequency, the size of -- - -- pixel-in the object is about 0.25 micron. For a 4 Mpixel camera (e.g. a Dalsa Falcon 4M30/60), running at 30 fps, with a pixel size of 7.4 microns, the magnification from the object to the 25th image generation camera is 7.4/0.25 = 30X. Therefore, a frame at 2352 x 1728 can cover a 0.588mm x 0.432mm area on the object, which equates to about 910 frames for a typical fabric section defined as 15mm x 15mm in area. The system described in the present document is desirably used where W
H ; —- T the tissue spatial variation in the focus dimension is much smaller than the frame size in the object. Variations in focus, in practice, occur over greater distances, and most focus adjustment is done to correct for skew. These slopes are generally in the range of 0.5 - 1 micron per 5 frame dimension on the object. The time to result for current scanning systems (e.g. a Biolmagene iScan Coreo system) is about 3.5 minutes for pre-scan and scan a 20x 15 mm x 15 mm field and about 15 minutes for a 40x scan on the 15mm x 15mm frame. The 15mm x 15mm frame is scanned by running 35 frames in 26 steps. Scans can be done unidirectionally with a retrace time of 1 second. The time to scan, using a technique according to the system described in the present document, can be about 5 seconds to find the nominal focus plane, 1.17 seconds per step (25 15 steps), for a total of 5 + 25 x (1.17 + 1) = 59.25 seconds (from about 1 minute). This is a considerable time saver over conventional approaches. Other embodiments of the systems described herein may allow for even faster focus times, but a limitation may occur in the amount of light needed for short lighting times to avoid motion blur in continuous scanning. Pulsing or strobing the light source 118, which "may be an LED light source, as discussed elsewhere in this document, to allow for high-peak illumination can mitigate this issue. In one embodiment, the pulsation of the light source 118 can be controlled by electronic focusing component 25 and control system 170. Additionally, running the system bidirectionally would eliminate the retrace time saving by about 25 seconds for a 20x scan which results in a scan of 35 seconds.
Q^
0
It should be noted that components used in connection with the focusing electronics and control system 170 may also,
more generally be called electrical components used to perform a variety of different functions in connection with the realizations of
5 techniques described in this document.
Figure 4 is a schematic illustration showing the oscillator focus stage 150 in more detail, in accordance with an embodiment of the system described herein.
The oscillator focus stage 150 may include an oscillator focusing lens 151 which can be moved by one or more
10 plus actuators 152a, b, such as moving coil actuators, and which may be mounted in a rigid housing 153. In one embodiment, the lens may be an achromatic lens that has a focal length of 500 mm, as is commercially available, see , for example, Edmund Scientific,
NT32-323. Alternatively, the oscillator focusing lens 151 can be
15 constructed from plastic, spherical and shaped in such a way that the weight of the lens is reduced (extremely low mass). a bending structure
154 may be attached to rigid housing 153 and secured to a point of rigid ground and may only permit translational movement of the oscillator focusing lens 151, e.g. short distances of about 600 to 1,000
20 microns.
In one embodiment, the flex structure 154 may be constructed of suitable stainless steel sheets, about 0.010" thick in the bending direction and forming a four-bar linkage.
Flexion 154 may be designed from a suitable steel spring, at a working stress far from its fatigue limit (factor of 5 below) to operate at
25 many cycles.
The moving mass of the oscillator 151 focusing lens and the bending
154 can be designed to provide about a 60 Hz or higher first mechanical resonance.
The moving mass can be monitored with a sensor of B
#
suitable high bandwidth position (eg " 1 KHz) 155, as a capacitive sensor or eddy current sensor, to provide feedback to the control system 170 (see Figure 2). For example, the KLA division Tencor's ADE manufactures a 2805 sensor probe
5 capacitive 5 mm with a bandwidth of 1 KHZ, measuring range of 1 mm and 77 nanometer resolution suitable for this application.
The oscillator focus and control system, as represented by "feature" included in element 170, can maintain "the" focusing lens amplitude of oscillator 151 within a prescribed focus range.
The oscillator focus and the
10 control can be based on well-known gain-controlled oscillator circuits.
When operated at resonance, the oscillator focusing lens 151 can be driven at low current, dissipating low power in the moving coil windings.
For example, using a BEI Kimco LAO8-10 (Winding A) actuator, the average currents can be less than
15 180 mA and the power dissipated may be less than 0.1 W.
It is noted that other types of oscillator lens movement and other types of actuators 152a,b may be used in connection with various embodiments of the system described herein.
For example, piezoelectric actuators can be used like actuators 152a, b.
In addition
In addition, the movement of the oscillator lens may be movement at non-resonant frequencies that remain independent of the movement of the microscope objective 120. Sensor 155, like the capacitive sensor mentioned above, as may be included in an embodiment according to the techniques of
25 herein, may provide feedback as to where the oscillator's focusing lens is positioned (e.g., relative to the sine wave or cycle corresponding to lens motions). As will be described elsewhere in this document, a determination may be W
25
--" 6 made so that the image frame obtained using the focus sensor produces the best sharpness value.
For this frame, the position of the oscillator focusing lens can be determined relative to the sine wave position as indicated by sensor 155. The position as indicated by
5 sensor 155 can be used by the control electronics 170 to determine an appropriate setting for the slow focusing stage 140,
For example, in one embodiment, the movement of the microscope objective_120 may be controlled by a slow-stage, slow-focus stage motor.
140, The position indicated by sensor 155 can be used to determine a
10 corresponding amount of movement (and corresponding control signal(s)) to position the microscope objective 120 in a better focus position in the Z direction.
The control signal(s) may be transmitted to the stepper motor of the slow focus stage 140 to cause any necessary repositioning of the microscope objective 120 to the
15 best focus position.
Figures 5A to 5E are schematic illustrations showing an iteration of focusing operations, according to the system described herein.
The figures show the image sensor 110, the focus sensor 160, the oscillator focusing stage 150 with an oscillator lens
20 and microscope objective 120, Tissue 101 is illustrated moving on the y-axis, i.e., XY motion stage 130, while focus operations are
- carried out.
In one example, the oscillator focusing stage 150 can move the oscillator lens at a desired frequency, such as 60 Hz or more (e.g., 80Hz, 100Hz), although it is noted that in other embodiments, the
The system described herein may also operate with the oscillator lens moving at a lower frequency (eg, 50Hz) according to applicable circumstances.
The XY 130 motion stage can be commanded to move, for example, in the Y direction, from the frame to the ü P
Õ adjacent frame. For example, stage 130 can be commanded to move at a constant 13 mm/sec. which for a 20x lens corresponds to an acquisition rate of about 30 frames/sec. Since the Oscillator Focus Stage 150 and XY Motion Stage 130 can be locked by 5 phase, the Focus Oscillator Stage 150 and Sensor 160 can do 60 focus calculations per second, or operate bidirectionally (which reads on the up and down motion of the sine wave) 120 focus points per second or 4 focus points per frame. For a frame height of 1728 pixels this equates to one focus point every 432 pixels or for the 20x objective every 10 108 microns. Since the XY motion piatina 130 is mobile, the focus point must be captured in a very short period of time, eg 330 µsec (or less), to keep variation in the scene to a minimum.
In various embodiments, as discussed elsewhere in this document, this data may be stored and used to extrapolate the next frame focus position, or alternatively, extrapolation may not be used and the last focus point is used to the focus position of the active frame. With an oscillator frequency of 60 Hz and a frame rate of 30 frames per second, the focus point is taken at a position of no more than 1/4 of a frame from the center of the 20 frame captured instantly. Generally, fabric heights don't change enough by 1/4 of a frame to make this focus point inaccurate.
A first focus point may be verified on the fabric to establish the nominal focus plane or reference plane 101'. For example, the reference plane 101' can be determined by initially moving the microscope objective 120, using the slow focus stage 140, through the entire Z range, say +1/-1 mm, and monitoring the sharpness values. Once the reference plane 101' is verified, the fabric 101 can
» be positioned in X and Y to start at a corner, and/or other particular location of the area of interest, and the oscillating focusing plate 150 is adjusted to move, and/or otherwise, the movement of the focusing plate oscillator 150 continues to be monitored, starting with Figure 5A.
5 The oscillator focus stage 150 can be synchronized with a master clock in the control system 170 (see Figure 2) which can also be used in connection with speed control of the XY movement stage 130. For example, if the stage of focus oscillator 150 moved through a sinusoidal motion from (peak to valley) pv of 0.6 10mm at 60 Hertz, assuming a 32% duty cycle to use the most linear winding range, 8 points could be collected across the focus range for a period of 2.7 msec. In Figures 5B to 5D, the oscillator focusing stage 150 moves the oscillator lens in a sinusoidal motion and focus samples are taken across at least a portion 15 of the sinusoidal curve. Focus samples would therefore be taken every 330 µsec or in a range of 3 KHZ. With a 5.5x magnification between the object and the focus sensor 160, a movement in the oscillator lens of 0.6 mm p-v equals a movement of 20 microns in the objective lens. This information is used to convey the position where the highest sharpness is computed, that is, the 20 best focus, to the slowest stepper motor of the slow focus stage 140. As shown in Figure 5E, the slow focus stage 140 is commanded to move the microscope objective 120 to the best focusing position (illustrated by 120' motion range) in time for the image sensor 110 to capture the best focus image 110' of the tissue area of interest 101.
In one embodiment, the image sensor 110 may be driven, for example, by the control system 170, to capture an image after a specified number of cycles of oscillator lens movement. The XY 130 motion stage moves to the next frame, the cyclic motion of the W
The oscillator lens on the oscillator focus stage 150 continues, and the focusing operations of Figures 5A to 5E are repeated. Sharpness values can be calculated in a range that does not congest the process, for example 3 KHZ.
Figure 6A is a schematic illustration of a plot 200 showing the oscillator focus optics drive waveform and sharpness determinations, in accordance with an embodiment of the system described herein. In an embodiment based on the times discussed in connection with the example of Figures 5A to 5E: 10 T = 16.67 msec, /* period of oscillator lens sinusoid if lens resonates at 60 Hz * F = 300 µm, / * positive range of focus values * N = 8, r number of focus points obtained in period E */ At = 330 µsec, I* the point of focus samples obtained every 330 15 µsec */ E = 2.67 msec, /* the period at which the N focus points are obtained */ Af = 1.06 µm at the center of the focus path. /* focus curve step size */ 20 Therefore, with this 32% duty cycle, 8.48 µm (8 x 1.06 µm = 8.48 µm) is sampled through focus processing.
Figure 6B is a schematic illustration showing a plot 210 of calculated sharpening values (Zs) for a portion of the oscillator lens sine wave motion shown in plot 210. The 25 position (z) for each plane of focus taken as sample as a function of each point i is given by EQUATION 1: z = Fcos [2n [('_:") + At · i] ;j EQUATION 1 By opening the window downwards, a CCD camera can and
W 0 will provide a high frame rate suitable for the system described herein. For example, the Dalsa company of Waterloo, Ontario, Canada produces the Genie M640-1/3 640 X480 monochrome camera. The Genie M640-1/3 will operate at 3000 frames/sec. at a frame size of 640 x 5 32. The pixel size in the CCD array is 7.4 microns. At 5.5x magnification between the object and the focus plane, one focus pixel is equivalent to about 1.3 microns on the object. Although some average of about 16 object pixels (4x4) per focus pixel may occur, enough spatial frequency contrast shift is preserved to obtain good focus information. In one embodiment, the best focus position can be determined according to the peak value of the sharpness calculation plot 210. In further embodiments, it is noted that other focus calculations and techniques can be used to determine the best position according to other metrics, including the use of a contrast metric, as discussed 15 elsewhere in this document. Figures 7A and 7B are schematic illustrations showing focusing determinations and adjustments of a specimen (tissue) in accordance with an embodiment of the system described herein. In Figure 7A, illustration 250 is a view of the specimen shown in close-up image frames 20 in connection with movement of the specimen along the Y axis, in accordance with movement of the XY motion stage 130 discussed herein. A traverse or step over the specimen in connection with movement of the specimen along the Y axis (e.g., according to movement of the XY stage) is illustrated at 250. Illustration 250' is an enlarged version of a portion of the illustration. 250. A frame of illustration 250' is designated dtp, in reference to a defined tissue point of the specimen. In the example in the 250' illustration, a specimen boundary is shown and, while scanning over it, multiple focus calculations are performed.
W 0 performed, according to the system described in the present document. In frame 251 and by way of example it is illustrated that a better focus determination is made after 4 focus calculations (shown as focus positions 1, 2, 3 and 0*) are performed in connection with imaging the specimen, 5 although further focus calculations can be performed in connection with the system described in this document. Figure 7B shows a schematic illustration 260 showing a plot of the position of the Z axis axis of the microscope objective in relation to the position of the Y axis of the specimen being examined. The illustrated position 261 shows the position 10 determined along the geometric axis Z to adjust the microscope objective 120 to achieve the best focus, in accordance with an embodiment of the system described herein.
It should be noted that the system described herein provides significant advantages over conventional systems such as those described in US Patent Nos. 7,576,307 and 7,518,642, which are hereby incorporated by reference, wherein the entire microscope objective is moved across the focus in a sinusoid or triangular pattern. The system provided herein is advantageous in the sense that it is suitable for use with the microscope objective and an accompanying stage which are heavy (especially if other objectives are added via a turret) and cannot be moved at frequencies highest values described using oscillator optics. The oscillator lens described herein can be mass-adjusted (eg, made lighter, less glass) and the imaging demands on the focus sensor are less than those imposed by the microscope objective. Focus data can be taken at high rates, as described in this document, to minimize scene variation when computing sharpness. By minimizing scene variation, the system described in
This document reduces discontinuities in the sharpness metric as the system moves in and out of focus while tissue is moving under the microscope objective. In conventional systems, such discontinuities add noise to the best focus calculation.
5 Figure 8 is a schematic illustration 300 showing an example of a sharpening profile, produced from movement through focus positions, including a sharpening curve and contrast ratio for each sharpening response at multiple points, which are sampled by the oscillator focusing optics, in accordance with an embodiment of the system described herein. Plot 310 shows the oscillator lens amplitude in micrometers on the x-axis and the sharpness units along the y-axis. As illustrated, the motion of the oscillator lens can be centered at representative points A, B, C, D, and AND; however, it has been observed that the computations described herein 15 can be applied to each of the points on the sharpening curve. The sharpening response produced from the focus sensor 160, per half cycle of the oscillator lens sinusoid, when the oscillator lens movement is centered at each of points A, B, C, D, and E is shown, respectively. , in plots 310a through e. Based on this, the contrast ratio for each of the 20 sharpening responses that has one of the corresponding points A to E is computed according to: Contrast function = (max - min.)/(max + min.). In connection with the contrast function determined for one of the points A to E (for example, where the oscillator lens movement is centered) and one of the corresponding sharpening response curves 310a to e, max represents the highest value of sharpness obtained from the sharpness response curve and min. represents the smallest sharpening value obtained from the sharpening response curve. The resulting contrast function plot 320 is shown below the sharpness curve plot 310 and the contrast ratio values of V
P 0 plot corresponding to oscillator lens movement, according to oscillator lens amplitude. The minimum of the contrast function in the 320 plot is the best focus position. Based on the contrast function and determination of the best focus position, a control signal can be generated 5 and is used to control the slow focus stage 140, to move the microscope objective 120 to the best focus position before the sensor 110 captures the image 110'. Figure 9 shows a functional control circuit block diagram 350 illustrating the use of the contrast function to produce a control signal 10 to control the slow focus stage 140, Ud can be regarded as a disturbance to the control circuit focus and can represent blade tilt or changing tissue surface heights, for example. Function block 352 shows the generation of sharpening vector information that can be generated by the focus sensor 160 and communicated to the focus electronics and control system 170. Function block 354 shows the generation of a contrast number ( e.g. value of the contrast function) at the point where the oscillator lens is sampling focus. This contrast number is compared to a set point or reference value (Ref.) produced in an initial step where the best focus 20 was previously established. The error signal produced from this comparison with the appropriate applied gain Ki (in function block 356) corrects the slow focus engine acting (in function block 358) to keep the scene in focus. It should be noted that one embodiment can adjust the position of the microscope objective 120 according to a minimum amount or threshold of movement. Thus, such a modality can avoid making adjustments smaller than the threshold.
Figure 10 is a schematic illustration showing the focus window 402 being broken into zones in connection with image processing.
P 33
V 0 —..
¥ . .. . ,, ._ focus, according to an embodiment of the system described herein. In the illustrated embodiment, the focus window is subdivided into 8 zones (402'); however, less or more than 8 zones can be used in connection with the system described in this document. A first 5 subset of zones can be within snapshot n and a second subset of zones are within snapshot n + 1. For example, Zones 2, 3, 4, 5 are within image frame 404 captured instantaneously at time tl . Zones 6 and 7 may be completely within the next image frame to be captured instantly as the 10 XY 130 motion stage traverses from the bottom to the top in the figure and/or Zones 0 and 1 may be completely within the next image frame to be captured instantly as stage 130 traverses from the top to the bottom of the figure. Focus positions 0, 1, 2, and 3 can be used to extrapolate the best focus position to the next frame captured instantly at position 0*. Fabric coverage can be established, for example, by running a serpentine pattern across the entire area of interest.
The rectangular window 404 of the image sensor can be oriented in the direction of travel of the stage 130, as a column of frames acquired during imaging 20 is aligned with the rectangular focus window 402. The size of the object in the image frame 406 , using, for example, a Dalsa 4M30/60 CCD camera, is 0.588 mm x. 0.432mm with the use of a 30X magnification tube lens. The size of the array can be (2352 x 7.4 microns/30) x (1720 x. 7.4 microns/30). The widest dimension of the image frame 406 (0.588 mm) can be oriented perpendicular to the focus window 402 and allows the minimum number of columns to traverse the tissue section. The focus sensor is 0.05mm x. 0.94 mm using 5x magnification on focus leg 406. Rectangular window 402 can be (32 x. 7.4 m G_
micron/5.0) x (640 x 7.4 micron/5.0). Therefore, the frame 402 of the focus sensor can be about 2.2x higher than the frame 404 of the image sensor, and can be advantageously used in connection with the multi-zone looking-ahead focusing technique, as already discussed
5 elsewhere in this document.
According to one embodiment of the system described herein, 120 best focus determinations can be made per second, with a sharpening calculation done every 333 µsec, which results in sharpness 8 calculated at 2.67 msec equal to one cycle of approximately 32% for an oscillator half-time of 8.3 msec of the
10 Oscillator lens movement.
A sharpening metric for each zone can be computed and stored.
When computing the sharpening metric for a single focus point, using multiple zones, the sharpening metric can be determined for each zone and combined, for example, as by adding all the
15 sharpness metrics for all zones considered at such a single point.
An example of the zone sharpening computation is shown in EQUATION 2 (eg, based on using a windowed camera for a 640 x 32 range). For row i, dimension n up to 32, and column j, dimension m up to 640/Z, where z is the number of zones, the sharpness for a zone can be represented
20 by EQUATION 2: Sharpness = E7 0' E7'6'"'[(Ii,j - Ii,j+k)'] EQUATION 2 where k is an integer between or equal to 1 and 5. Other metrics of and sharpening algorithms may also be used in connection with the system described in this document.
As per XY movement plate
25 130 is moving along the Y axis, the system acquires sharpening information for all Zones 0 through 7 in the focus window 402. It is desirable that while the stage 130 is moving, to know how the section heights of fabric are varying.
Computing the sharpness curve
- Ü —-m .- e (maximum sharpness being the best focus), by varying the focus height, Zones 6 and 7, for example, can provide information before moving the next frame where the next best focus plane is positioned. If large shifts in focus are anticipated by this looking ahead, stage 130 can be delayed to provide closer spaced points to better drive the height transition.
During the scanning process, it can be advantageous to determine if the system is transitioning from a white space (in tissue) to a darker space (fabric). By computing the sharpness, in the 10 Zones 6 and 7, for example, it is possible to predict whether this transition is about to occur. While sweeping the column, if Zones 6 and 7 show increased sharpness, the XY Motion Stage 130 can be commanded to decelerate to create closely spaced focus points at the edge of the fabric. If, on the other hand, motion from high sharpness to low sharpness is detected, then it can be determined that the raster view is entering a white space, and it may be desirable to slow down the stage 130 to create spaced focus points. closest to the fabric boundary. In areas where these transitions do not occur, the stage 130 can be commanded to move at higher constant speeds to increase the overall blade sweep throughput. This method may advantageously allow rapid tissue scanning. According to the system described in this document, snapshots can be taken while focusing data is collected. In addition, all focus data can be collected on a first scan and stored, and snapshots can be taken at the best focus points during a subsequent scan. One embodiment may use contrast ratio or function values in a manner similar to that described herein with sharpness values to detect changes in focus and therefore determine the k—
& - and transitions into or out of areas that contain fabric or white space.
For example, for a 20X 15mm x 15mm raster, at 0.588 x 0.432mm image frame size, there are 26 columns of data, each column has 35 frames. At an imaging rate of 30 fps, each column is traversed in 1.2 seconds or a scan time of about 30 seconds. Since the focus sensor 160 computes 120 (or more) focus points per second, the system described herein can achieve 4 focuses per frame (120 focuses/sec. divided by 30 fps). At an imaging rate of 60 fps, the scan time is 15 seconds and 210 focuses per frame (120 focuses/sec divided by 60 fps). In another embodiment, a color camera can be used as the focus sensor 160 and a chroma metric can be determined alternatively and/or in addition to the sharpness contrast metric. For example, a Dalsa color version of the Genie 640 x 480 camera can be properly used as the 160 focus sensor according to this realization. Metric chroma can be described with polychromy in relation to the brightness of a similarly lit white. In equation form, (EQUATIONS 3A and 3B), the chroma (C) can be a linear combination of the color measurements of R, G, B: 20 Cb=-37.797xR-74.203xG+112xB EQUATION 3A Cr=112xR- 93.786xG-18.214xB EQUATION 3B Note that R=G=B, CB = CR =0. A value for C, which represents total chroma, can be determined based on Cb and Cr. (eg as per addition of Cb and Cr).
25 As the XY motion stage 130 moves along the y axis, the focus sensor 160 can acquire color information (R, G, B) as in a bright field microscope. It is desirable for the stage to be moving to know how the fabric section heights are
— 37 'W' »
G varying. The use of RGB color information can be used, as with the contrast technique, to determine whether the system is transitioning from white space (in fabric) to colored space (fabric). By computing chroma in Zones 6 and 7, for example, it is possible to predict whether this transition is about to occur. If, for example, very small chroma is detected, then C=O and it can be recognized that no tissue boundaries are approaching.
However, while scanning the focus column, if Zones 6 and 7 show increased chroma, then stage 130 can be commanded to slow down to create more closely spaced focus points on the 10 tissue boundary. If on the other hand high chroma movement to decrease chroma is detected, then it can be determined that the digitizer is entering a white space, and it may be desirable to slow down the stage 130 to create more closely spaced focus points at the boundary. of fabric. In areas where such transitions do not occur, stage 130 may be commanded to move at constant higher speeds to increase overall blade sweep throughput.
In connection with the use of sharpness values, contrast ratio values, and/or chroma values to determine when the field of view or next frame(s) is entering or leaving a tissue-bearing blade area, 20 variations of processing can be done. For example, when entering a fabric area of white space (for example, between fabric areas), movement in the Y direction may be decreased and the number of focus points obtained may also increase. When viewing white space or an area between tissue samples, movement in the Y direction can be increased and fewer focus points determined until movement over an area containing tissue is detected (e.g., as by increased chroma and/or of sharpness).
Figure 11 shows a graphical illustration of different values of
W « 38 --. it .. _-_ ". _ ,. _ 0
W sharpening that can be obtained at points in time in a realization in accordance with techniques in this document. The upper portion 462 includes a curve 452 corresponding to a sine wave half cycle (e.g., a single peak half to peak cycle or period) of oscillator lens movement.
5 The X axis corresponds to oscillator lens amplitude values during this cycle and the Y axis corresponds to sharpness values.
- Each of the points, like point _462.a, represents a point at which a frame is taken using the focus sensor where each frame is taken on a lens oscillator amplitude represented by the point's X10 geometry axis value and has a sharpness value represented by the point's Y-axis value. Element 465 in lower portion 464 represents a curve adapted to the setting of sharpness values obtained as depicted in portion 462 for the illustrated data points. Figure 12 is a flow diagram 500 showing real-time focus processing during scanning of a specimen under examination in accordance with an embodiment of the system described herein. In step 502, a nominal focus plane or reference plane can be determined for the specimen being examined. After step 502, processing proceeds to a step 504 where an oscillator lens, 20 in accordance with the system described in this document, is configured to move at a particular resonant frequency. After step 504, processing proceeds to a step 506 where the XY motion plate is commanded to move at a particular speed. It is noted that the order of steps 504 and 506, like other processing steps discussed in this document, can be appropriately modified in accordance with the system described in this document. After step 506, processing proceeds to a step 508 where sharpening calculations for focus points with respect to the specimen being examined are performed in connection with ab 0 m motion (e.g. sinusoidal) of the oscillator lenses according to the system described in this document. Sharpness calculations may include use of contrast, chroma and/or other appropriate measurements as further discussed elsewhere in this document.
5 After step 508, processing proceeds to a step 510 where a better focus position is determined to position a microscope objective used in connection with an image sensor to capture an image according to the system described" in this document.
After step 510, processing proceeds to a step 512 where a control signal concerning the best focus position is sent to a slow focus stage which controls the position (Z axis) of the microscope objective. Step 512 may also include sending a trigger signal to the camera (e.g., image sensor) to capture an image of the specimen portion under the objective. The trigger signal may be a control signal that causes the image to be captured by the image sensor, for example, after a specific number of cycles (for example, as related to oscillator lens movement). After step 512, processing proceeds to a test step 514 where it is determined whether the speed of the XY motion stage, which holds the specimen under scan, should be adjusted. The determination can be made in accordance with view head processing techniques using sharpening and/or other information from multiple zones in a field of view focus, as further discussed in detail elsewhere in this document. If, in test step 514, it is determined that the speed of the XY stage is to be adjusted, then processing proceeds to a step 516 where the speed of the XY movement stage is adjusted. After step 516, processing returns to step 508. If, in test step 514, it is determined that no adjustment to the speed of the XY motion stage b 4 " '"a- - — is to be made, then processing continues to a test step 518 where it is determined whether focus processing should continue. If processing continues, then processing returns to step 508. Normally, if processing does not continue (eg scanning of the current 5th specimen is complete), then focus processing is terminated and processing is complete.
Figure 13 is a flow diagram 530 showing processing on the slow focus stage in accordance with an embodiment of the system described herein. At step 532, the slow-focus stage, which 10 controls a position (e.g., along the Z-axis axis) of a microscope objective, receives a control signal with information to adjust a position of the microscope objective it is examining. a specimen. After step 532, processing proceeds to a step 534 where the slow focus stage adjusts the position of the 15° microscope objective in accordance with the system described herein. After step 534, processing proceeds to a hold step 536 where the slow focus stage waits to receive another control signal. After step 536, processing returns to step 532. Figure 14 is a flow diagram 550 showing image capture processing in accordance with an embodiment of the system described herein. In step 552, an image sensor of a camera receives a signal! trigger and/or other instruction that triggers processing to capture an image of a specimen under microscopic examination. In various embodiments, the trigger signal may be received from a control system that controls triggering of image sensor image capture processing after a number of specific cycles of motion of an oscillator lens used in focus processing in accordance with the system described in this document. Alternatively, the m
F 41 u
W -e— trigger signal can be provided based on a position sensor on the XY motion plate. In one embodiment, the position sensor may be a Renishaw Linear Encoder Model No. T1OOO-10A. After step 552, processing proceeds to a step 554, where the image sensor 5 captures an image. As discussed in detail in this document, the image captured by the image sensor may be in focus in connection with operation of a focusing system in accordance with the system described in this document. Captured images can be stitched together using other techniques referenced in this document. After step 554, processing proceeds to a step 556 where the image sensor waits to receive another trigger signal. After step 556, processing goes back to step 552.
Figure 15 is a schematic illustration 600 showing an alternative arrangement for focus processing according to an embodiment of the system described herein. A windowed focus sensor may have a Field Of View (FOV) 602 frame that can be tilted or normally positioned to diagonally sweep a range substantially equal to the width of the FOV 604 imaging sensor frame. described in this document, the window can be tilted in the direction of travel. For example, the 602 frame FOV of the tilt focus sensor can be rotated 45 degrees which would have an effective width of 0.94 x 0.707"" 0.66mm on the object (fabric). The imaging sensor's 604 frame FOV can have an effective width of 0.588mm, so as the XY motion plate holding the tissue moves under the lens, the 602 tilt focus sensor's frame FOV checks the edges of the range observed by the image sensor. In the view, multiple frames from the tilted focus sensor are shown superimposed on the image frame FOV of sensor 604 at intermediate positions at times 0, 1, 2, and 3.
" 'e' "" " Focus points can be employed at three points between the centers of adjacent frames in the focus column. Focus positions 0, 1, 2, and 3 are used to extrapolate the best focus position to the next frame captured at position 0*. The scan time for this method should be similar to the method described elsewhere in this document. While the 602 frame FOV of the tilt focus sensor has a shorter look ahead, in this case 0.707 x (0.94-0.432)/2 = 0.18mm or the tilt focus sensor invades 42% in the next frame to be acquired, the 602 frame FOV of the tilted focus sensor, which is oblique to the 604 image sensor 10 frame FOV, scans tissue at the edges of the scan range which can be advantageous in certain cases to provide information focus icon.
Figure 16 is a schematic illustration 650 showing an alternative arrangement for focus processing in accordance with another embodiment of the system described herein. As in illustration 650, the 652 frame FOV of the tilt focus sensor and the 654 frame FOV of the image sensor are shown. The tilt sensor's 652 frame FOV can be used to acquire focus information on the forward pass through tissue. In the back pass, the imaging sensor 20 captures frames while the focus stage is adjusted using the previous forward pass focus data. If one wants to get focus data at every intermediate image frame skip position 0, 1, 2, 3 in the previous method, the XY motion stage can move 4x speed in forward pass given the high rate of acquisition of focus point. For example, for 15mm x 15mm at 20X, a column of data is 35 frames. Since focus data is acquired at 120 points per second, the pass-through can be performed in 0.3 seconds (35 frames/120 focus points per second). The number of columns in this example is 26, so k r
oh 43
W d the focus portion can be done at 26 x 0.3 or 7.6 seconds. Image acquisition at 30 fps is about 32 seconds. Thus, the focus portion of the total scan time is only 20°/o, which is efficient. Also, if focus was allowed to jump at every other frame, the focus portion of the 5-sweep time must additionally drop substantially.
VE noted that in other embodiments, the focus range of the focus sensor can be positioned at other locations in the field of view, and in other orientations, to sample adjacent columns of data to provide additional looking-ahead information that can be 10 used in connection with the system described in this document. The XY motion plate that transfers the blade can repeat the best focus points produced on the forward stroke over those produced on the previous stroke. For a 20x 0.75 lens where a depth of focus is 0.9 micron, it will be desirable to repeat down to about 0.1 15 micron. Stages can be constructed so that they encounter 0.1 micron of forward/reverse repeatability and consequently this requirement is technically possible, as further discussed elsewhere in this document.
In one embodiment, a tissue or smear on a glass slide being examined according to the system described herein may cover the entire slide or an area of approximately 25 mm x 50 mm. Resolutions are dependent on objective numerical aperture (NA), blade coupling medium, condenser NA, and light wavelength. For example, at 60x, for a 25 microscope objective of 0.9 NA, the apochromatic plane (Plan APO), in air in green light (532 nm), the side resolution of the microscope is about 0.2 µm with a depth of focus of 0.5 µm. In connection with operations of the system described in this
& 4 document, digital images can be obtained by moving a limited field of view via a CCD array or line scan sensor over the area of interest and assembling the limited fields of view or frames or tiles together to form a mosaic. It is desirable for the mosaic to appear seamless with no visible seams, focus anomalies, or radiance as the viewer navigates through the entire image.
Figure 17 is a flow diagram 700 showing processing to acquire a tissue mosaic image on a slide in accordance with an embodiment of the system described herein.
10 At step 702, the thumbnail image of the blade can be acquired. The minimized image can be a low resolution on the order of lx or 2x magnification. If a barcode is present on the slide label, the barcode can be decoded and affixed to the slide image at this step. After step 702, processing proceeds to a step 704 where tissue 15 can be found on the slide using standard image processing tools. The tissue can be delimited to narrow the scan region to a given area of interest. After step 704, processing proceeds to a step 706 where an XY coordinate system can be attached to a fabric surface. After step 706, processing may proceed to a step 708 where one or more focus points may be generated at regular X and Y spacing for the tissue and best focus may be determined using a focusing technique such as one or more of the following: real-time focusing techniques discussed elsewhere in this document. After step 708, processing may proceed to a step 710 where desired focus point coordinates, and/or other appropriate information, may be saved and may be referred to as anchor points. It is noted that where frames rest between anchor points, a focus point can be tweened.
Ò After step 710, processing can proceed to a step 712 where the microscope objective is positioned to the best focusing position according to the techniques discussed elsewhere in this document.
After step 712, processing proceeds to a step 714 where an image is collected. After step 714, processing proceeds to a test step 716 where it is determined whether an entire area of interest has been scanned and imaged. If not, then processing proceeds to a step 718 where the XY stage moves the tissue in the X direction and/or Y direction in accordance with techniques discussed elsewhere in this document. After step 718, processing returns to step 708. If in test step 716, it is determined that an entire area of interest has been scanned and imaged, then processing proceeds to step 720 where the image frames collected are stitched or normally combined together to create the mosaic image according to the system described in this document and using techniques discussed elsewhere in this document (referring, for example, to Patent Application Publication No. US 2008/0240613 ). After step 720, processing is complete. It is noted that other suitable sequences may also be used in connection with the system described in this document to acquire one or more mosaic images. For advantageous operation of the system described in this document, the positional repeatability z may be repeatable for a fraction of the objective depth of focus. A small error in returning to the z position by the focus motor is easily seen in a tiled system (2D CCD or 25 CMOS) in adjacent columns of a line scan system. For the above mentioned resolutions of 60X, a z-peak repeatability on the order of 150 nanometers or less is desirable, and such repeatability should consequently be suitable for other objectives, such as 4x objectives, B U.
20x and/or 40x. In accordance, further, with the system described herein, various embodiments for a slide stage system that includes an XY stage are provided for pathology microscopy applications that can be used in connection with imaging resources and techniques. of digital pathology that are discussed in this document, which includes, for example, functioning as the XY motion stage 130 discussed elsewhere in this document in connection with "real-time focusing" techniques. According to one embodiment, and as further discussed in detail elsewhere in this document, an XY stage may include a compact base block. The base block may include a flat glass block supported on raised ledges and a second glass block having a triangular cross-section supported on raised ledges. The two blocks can be used as straight and flat rails or ways to guide a platinum block of movement. Figure 18 is a schematic illustration showing an implementation of a precision stage 800 (e.g., a Y stage portion) of an XY stage according to an embodiment of the system described herein. For example, 800 precision platinum can achieve z-peak repeatability on the order of 150 nanometers or less over a 25 mm x 50 mm area. As further discussed elsewhere in this document, the Precision Stage 800 may be used in connection with features and techniques discussed elsewhere in this document, which includes, for example, operation in connection with the XY Motion Stage 130 25 discussed in relation to to real-time focusing techniques. Precision stage 800 may include a compact base block 810 where a flat glass block 812 is supported on raised ledges. The spacing of these projections is such that the slope, due to the weight of the
B the 800 precision of the glass blocks in the simple supports is minimized.
A second glass block 814 having a triangular cross-section is supported on raised ledges.
The glass blocks 812, 814 can be adhesively bonded to the base block 810 with a semi-rigid epoxy that does not cause
5 pressure to the glass blocks.
Glass blocks 812, 814 can be straight and polished to one or two light waves at 500 nm.
A low thermal expansion material such as Zerodur may be employed as a material for the glass blocks 812, 814. Other suitable types of glass may also be used in connection with the system described herein.
A cut
10 816 can allow light from a condenser microscope to illuminate tissue on the slide.
The two glass blocks 812, 814 can be used as straight, flat rails or ways to guide a motion platen block 820. The motion platen block 820 may include ball-shaped knobs of
15 rigid plastic (eg, 5 buttons) that make contact with the glass blocks, as illustrated in positions 821a-e.
Because these plastic buttons are spherical, the contact surface can be confined to a very small area (<0.5 mm) determined by the elastic modulus of the plastic.
For example PTFE or other thermoplastic blend plus other additives
20 lubricants from GGB Bearing Technology Company, UK, can be used and launched in the shape of approx.
- mm in diameter.
In one embodiment, the coefficient of friction between the plastic knob and the polished glass should be as low as possible, but it may be desirable to avoid using a liquid lubricant to save on maintenance.
25 instrument.
In one embodiment, a coefficient of friction between 0.1 and 0.15 can be immediately achieved by dry activation.
The figures. 19A and 19B are more detailed views of the motion platen block 820 in accordance with an embodiment of the described system.
This document shows the spherically formed buttons 822a-e that contact the glass blocks 810, 812 at positions 821a-e. The knobs can be arranged in positions that allow for excellent rigidity in all directions other than the drive direction (Y). For example, 5 two plastic buttons may face each other to contact the sides of the triangular shaped glass block 814 (i.e. 4 buttons 822b-e) and a plastic button 822a is positioned to contact with the glass flat block 812. The platinum motion block 820 may include one or more holes 824 to be light weighted and are formed to lie 10 at the center of gravity at the centroid 826 of the triangle formed by the position of plastic support knobs. 822a-e. In this way, each of the plastic knobs 822a-e at the corners of the triangle 828 can be of equal weight at all times during motion of the platen 800.
Referring back to Figure 18, a blade 801 is secured by means of a spring loaded arm 830 to the blade shelter 832. The blade 801 may be manually installed in the shelter 832 and/or robotically installed in the shelter 832 with a mechanism assistant. A rigid cantilever arm 840 rigidly supports and secures the end of a small diameter bending rod 842 which may be made of high fatigue strength steel. In one example, this diameter might be 0.7 mm. The other end of the bending rod 842 may be attached to the centroid location 826 on the - -- --- motion plate 820. The cantilever arm 840 may be attached to a bearing block 850 which may function by means of a recirculating bearing in a hardened rail steel 852. A lead screw assembly 25 854 may be attached to the bearing block 850 and the lead screw assembly 854 may be rotated by a stepper motor 856. Components suitable for the elements noted above may be be available through various corporations such as THK in
& " ·· * Japan. Lead screw assembly 854 drives bearing block 850 on rail 852 which pulls or pushes motion platinum block 820 via bending rod 842. The bending rigidity of bending rod 842 may be a factor of 5 greater than 6.0OOX less than the 820 motion platinum block stiffness in its plastic blocks (this is a stiffness that opposes a force orthogonal to the motion platinum plane of motion in the z-direction). This effectively isolates the platinum block from 820 motion and up and down motion of the 850 bearing block/840 cantilever arm produced by 10 bearing noises. The balance of mass and attention to geometry in the 800 precision platinum design described in this document minimizes moments in the motion platen block 820 that can produce small rocking motions. Additionally, since the motion platen block 820 operates on 15 polished glasses, the motion platen block 820 has repeatable z position age of less than 150 nanometers which magnifies enough to sweep at 60X magnification. Since the 60X condition is the most severe, other lower magnifications such as 20x and 40x high NA objectives also show adequate performance similar to the performance obtained under 60X conditions. Figure 20 shows an implementation of an entire XY composite stage 900 in accordance with the precision features stage discussed in this document and includes a Y stage 920, an X stage 940 and a base plate 960 in accordance with an embodiment of the system described herein. document.
25 In this case, a base block for the stage Y 920 becomes the stage X 940 which is a stage that moves in the X direction. A base block for the stage X 940 is a base plate 960 that can be fixed to the ground. The XY 900 composite platinum provides repeatability in the Z direction on the order of 150 nanometers and 6 Ü'
repeatability on the order of 1 to 2 microns (or less) in the X and Y directions according to the system described in this document.
If the platins include position feedback via a range scale, such as those produced by Renishaw of GIoucestershire, England, subscale accuracies
5 microns are achievable according to the system described in this document.
The stage design according to the system described in this document can be superior to a sustained spherical bearing that moves stages where an XY stage according to the system described in this document does not suffer from repeatability errors due to ball bearings
10 non-spherical or non-cylindrical roller bearings.
Also, in non-rolling bearing designs, a new ball component, in different ball sizes, may cause non-repeatable motion.
An additional benefit of the achievements described in this document is the cost of platinum.
The glass elements use standard lapping and polishing techniques and are not
15 excessively expensive.
The bearing block and lead screw assembly need not be of particularly high quality in which the rod bending decouples a motion platen from the bearing block.
Additionally, according to the system described in this document, it is advantageous to reduce and/or normally minimize
20 scan while scanning digital pathology slides.
In clinical settings, a desirable workflow is to install a slide rack on a robotic slide scanning microscope, close the door, and command the system to scan the slides.
It is desirable that no user intervention is required until all slides are scanned.
The size
25 batch can include multiple slides (e.g. 160 slides) and the time to scan all slides is called the batch time.
The slide throughput is the number of slides per hour processed.
Cycle time is the time between each available blade image! that is ready for
V m 0 visualization. Cycle time can be influenced by the following steps in acquiring an image: (a) robotically obtaining the blade; (b) creating a minimized view or overview image of the slide and label tissue area; 5 (C) calculating an area of interest that borders the slide tissue; (d) pre-scanning the limited tissue area to find a regular array of better focused points on the tissue; (e) sweeping the tissue in accordance with movement of a stage and/or sensor; (f) create one. compressed output image ready for viewing; and (g) deposit the slide, ready for the next slide. It is noted that step (d) may not be necessary if dynamic focusing or "real-time" focusing is performed according to the system described in this document, in which the scan/image acquisition time can consequently be reduced. as a result of using real-time focusing techniques.
15 . The system described in the present invention may additionally involve eliminating or significantly shortening the time to perform steps (a), (b), (c) and (g). In accordance with various embodiments of the system described in the present invention, such gains can be realized, for example, using the storage-to-store concept in which the above-noted steps (a), (b), (C) and ( g) for one lamina are overlapped in time with steps (d), (e) and (f) for another lamina, as discussed further in detail in the present invention. In various embodiments, overlapping steps (a), (b) and (C) for one blade with steps (d), (e) and (f) for another blade can provide a gain of 10%, 25%, or 25 even 50% compared to a system where steps (a), (b) and (C) for one slide do not overlap with steps (d), (e) and (f) for another slide.
Figure 21 is a schematic illustration showing a blade storage device 1000 according to an embodiment
P q of the system described in the present invention.
A blade pickup head 1002 may be positioned to withdraw a blade 1001. The withdrawal head 1002 may use a mechanical device and/or a vacuum device to withdraw the blade 1001. The blade 1001 may be one of a collection of
5 slides in the batch, for example a batch of 160 slides.
The blade collection can be arranged on a blade holder 1003. Pickup head 1002 is attached to a bearing block or carriage 1004 which runs on a steel "rail" 1005. Bearing block 1004 is driven by a screw 1006 rotary feed. Motor counts can be detected with a
10 rotary encoder 1007 and converted to linear path to control the blade position in the Y direction.
Elements 1002 to 1007 may comprise a movable assembly referred to as a blade loader/unloader 1008. Blade loader/unloader 1008 may also move on a motorized rolling block or carriage 1009 in the x direction in the
15 rail 1010 which allows the blade loader/unloader 1008 to " move in both X and Y directions.
In operation, a blade, while still attached to the pickup head 1002, can be positioned under a low resolution camera 1011 to obtain the overview image or thumbnail view of the tag or
20 blade tissue area (e.g. step noted above (b)). Once this operation is completed, step (C) can be performed and the blade is placed in a position in a blade store 1012. The blade store 1012 may include two (or more) store positions or slots 1018a, 1018b and is shown including a blade 1017 in the
25 storage position 1018a.
In one embodiment, a composite XY stage 1013 may include a stage plate 1014 that moves in the Y direction and which is mounted on a plate 1015 that moves in the x direction.
XY 1013 platinum may have attributes and
There are functionality similar to those discussed elsewhere in the present invention, including, for example, attributes of the composite platinum XY 900 discussed in the present invention.
Platinum plate 1014 may additionally include an additional blade pickup head 1016. The
5 pickup head 1016 may be similar to pickup head 1002 described above.
Pickup head 1016 may use a mechanical device and/or a vacuum device to withdraw a blade.
The pickup head 1016 of the XY composite stage 1016 can move to the storage position 1018a and remove the blade 1017. A
10 blade 1017 can now proceed to one or more of the steps noted above, including steps: (d) pre-scan, (e) scan, and (f) create output image steps.
While this processing is running, the blade loader/unloader 1008 can take out another blade (e.g. blade 1001), get the thumbnail view of blade 1001 using
15 of camera 1011 and placing Blade 1001 in an empty position 1018b in blade store 1012, shown schematically by dotted line 1001'. When the scan is complete on the anterior blade (blade 1017), the blade pick-up head 1016 of the XY composite stage 1013 can place the blade 1017 in the storage position 1018a and withdraw the
20 next blade (blade 1001) of the storage position 1018b that is ready for scanning.
The 1013 XY Composite Stage can move in a regular back and forth scanning pattern under a 1019 High Resolution Optical System's camera and microscope optics to acquire a high definition image of the biological tissue according to the attributes and
25 techniques discussed elsewhere in the present invention.
It is further noted that the blade movements and selections of the XY composite stage 1013 and/or blade loader/unloader 1008 may be controlled by one or more processors in a control system.
Blade loader/unloader 1008 can move to storage position 1018a and withdraw blade 1017 and deposit blade 1017 in Blade holder 1003. This blade 1017 has completed all the steps enumerated above. The blade loader/unloader 1008 can then continue to withdraw and load another blade into the blade store 1012 and eventually withdraw and return the blade 1001 to the blade holder.
1003. Processing similar to that described above can continue until the slides that are in the slide holder 1003 are scanned.
Blade storage techniques according to the system described in the present invention provide advantageous time savings. For example, in a system with a 20x15mm x 15mm field, the recall time is about 25 seconds, the thumbnail acquisition is about 10 seconds, the prescan time is about 30 seconds, and the scan time is 90 seconds. Output file generation is done simultaneously with scanning process and can add about 5 seconds. Blade deposit is about 20 seconds. Adding all these times together gives a cycle time of 180 seconds. The XY composite stage still needs time to remove and deposit the swept slide which can be up to 10 seconds. Consequently, the reduction in scan time is therefore about 1-(180 - 55 + 10) /180 = 25%. For systems that use dynamic focusing techniques, such as real-time focusing as discussed further elsewhere in the present invention, pre-scan time can be eliminated and with high data rate cameras with times not associated with recall. and deposit can reduce by 20 to 30 seconds. The reduction in scan time when using blade storage in this case can be about 1- (75 - 55 + 10) /75 = 56°/o.
Figure 22A is a flowchart 1100 showing the processing
.
0 blade storage according to an embodiment of the system described in the present invention in connection with a first blade. In a step 1102, the first blade is removed from a blade holder. After step 1102, processing proceeds to a step 1104 where a thumbnail image is obtained and/or other thumbnail processing, which may include determining an area of interest of tissue on the slide, is performed for the first slide. After step 1104, processing proceeds to step 1106 where the first slide is deposited in a slide store. After step 1106, processing proceeds to a step 110810 where the first slide is taken from the slide store. After step 1108, processing proceeds to a step 1110 where the first slide is scanned and imaged according to techniques similar to those discussed further elsewhere in the present invention. It is noted that in various embodiments the scanning and imaging techniques 15 may include pre-scan focusing steps and/or that use dynamic focusing techniques, such as a real-time focusing technique.
After step 1110 processing proceeds to a step 1112 where the first slide is deposited in the slide store. After step 1112, processing proceeds to a step 1114 where the first slide is taken from the slide store. After step 1114, processing proceeds to a step 1116 where the first slide is deposited on the slide holder. After step 1116, processing is completed with respect to the first blade. Figure 22B is a flowchart 1120 showing blade storage processing 25 in accordance with an embodiment of the system described in the present invention in connection with a second blade. As discussed further in the present invention, various steps of flowchart 1120 may be performed in parallel with steps of e.
56 0 « K " "" " " " "
flowchart 1 100. In a step 1122, the second blade is removed from a blade holder.
After step 1102, processing proceeds to a step 1124 where a thumbnail image is obtained and/or other thumbnail processing, which may include determining an area of
5 interest of the tissue on the slide, is carried out for the second slide.
After step 1124, processing proceeds to step 1126 where the second slide is deposited in a slide store.
After step 1126, processing proceeds to a step 1128 where the second blade is removed from the blade store.
After step 1128 processing
10 proceeds to a step 1130 where the second slide is scanned and imaged according to techniques similar to those discussed further elsewhere in the present invention.
It is noted that in various embodiments the scanning and imaging techniques may include pre-scan focusing steps and/or that use focusing techniques.
15 dynamics, such as a real-time focusing technique.
After step 1130 processing proceeds to step 1132 where the second slide is deposited in the slide store.
After step 1132, processing proceeds to a step 1134 where the second blade is removed from the blade store.
After step 1134, processing
20 proceeds to a step 1136 where the second blade is deposited on the blade holder.
After step 1136, processing is completed with respect to the second blade. "
According to an embodiment of the system described in the present invention that is directed to slide storage, flowchart steps
25 1100 with respect to the first blade may be performed by a blade storage device in parallel with the steps of flowchart 1120 with respect to the second blade in order to reduce cycle time.
For example, steps 1122, 1124, 1126 of flowchart 1120 for second slide (for example, steps in connection with removing the second sheet from the slide holder, processing the thumbnail image, and depositing the second sheet in the storage slide) may overlap steps 1108, 1110, and 1112 of flowchart 1100 with respect to the first slide (e.g., the 5 steps in connection with taking the first slide out of the slide store, scanning and imaging the first slide, and deposit the first slide back into the slide store). Furthermore, steps 1134 and 1136 (e.g., steps in connection with withdrawing the second blade from the blade store and depositing the blade in the blade holder) may also overlap the steps of scanning the first blade. Time gains of up to 50°/o can be obtained according to parallel blade processing techniques according to the system described in the present invention compared to processing one blade at a time, with additional gains possible with the use other aspects of the system and techniques described in the present invention. Figures 23A and 23B show timing diagrams using blade storage techniques in accordance with the system embodiments described in the present invention and which illustrate time savings in accordance with various system embodiments described in the present invention.
Figure 23A shows the timing diagram 1150 for the situation where a prescan step is used. The timing diagram shows timing for three slides (Blades 1, 2 and 3) over a duration of approximately 300 seconds in connection with performing slide processing steps using slide storage including removing a slide from a slide holder, process thumbnail image, deposit slides in store, take out of store, pre-sweep, sweep slides and output files, deposit in store and deposit in slide holder. As illustrated, in
Ra—in one embodiment, the cycle time for the illustrated processing may be approximately 150 seconds.
Figure 23B shows the timing diagram 1160 for a situation where a real-time focusing technique is used (no prescan). The timing diagram shows timing for three blades (Blades 1, 2 and 3) over a duration of approximately 150 seconds in connection with performing steps of sweeping and moving the blade with the use of blade storage including pulling out a blade from a slide rack, process the thumbnail image, deposit slides into the rack, remove from rack, sweep slides and output files, drop into rack, and drop into slide rack. As illustrated, in one embodiment, the cycle time for the illustrated processing may be approximately 50 seconds. Figure 24 is a schematic illustration showing a blade storage device 1200 in accordance with another embodiment of the system described in the present invention. In the illustrated embodiment, no storage is required and deposit, thumbnail, and withdrawal times can be eliminated from the cycle time using blade storage device 1200. Blade storage device 1200 may include two XY composite stages. 1210, 1220 that operate independently. Each of the XY composite stages 1210, 1220 may have attributes similar to those discussed in the present invention with respect to the XY composite stage 1013. A first blade holder 1211 may be positioned at one end of the platen 1210 and a second blade holder 1221 may be positioned at one end of the blade 1220. It is noted that in connection with another embodiment of the system described in the present invention, the first blade holder 1211 and the second blade holder 1211 may refer instead to portions of a blade support.
d.
Two miniature cameras 1212, 1222 can serve each of the XY composite stages 1210, 1220. Each of the blade holders 1211, 1221 can serve blades to its concurrent XY composite stage 1210, 1220 with a corresponding pickup head. An optical microscope train 1230 5 can serve both XY composite stages 1210, 1220. For example, while one of the XY composite stages (e.g. 1210 stage) is sweeping a slide, the other (e.g. 1220 stage) is performing its deposit, miniature and withdrawal functions with another blade. These functions can be overlapped with scan time. Consequently, the cycle time can be determined by the scan time of a blade and the deposit, miniature and withdrawal times are therefore eliminated from the cycle time in accordance with the illustrated embodiment of the system described in the present invention. Figure 25A is a flowchart 1250 showing slide storage processing in connection with a first slide in accordance with a described embodiment of the system for a slide storage device having two XY composite stages for slide processing. In a step 1252, the first blade is removed from a blade holder. After step 1252, processing proceeds to a step 1254 where miniature processing is performed on the first slide. After step 1254, processing proceeds to a step 1256 where the first slide is scanned and imaged according to techniques similar to those discussed further elsewhere in the present invention. It is noted that in various embodiments the scanning and imaging techniques may include pre-scan focusing steps and/or that use dynamic focusing techniques, such as a real-time focusing technique. After step 1256, processing proceeds to step 1258 where the first slide is deposited back into the slide holder. After step 1258, processing is completed with respect to the
.
4 first blade. Figure 25B is a flowchart 1270 showing slide storage processing in connection with a second slide in accordance with an embodiment of the described system for a slide storage device having two XY composite stages for slide processing. In a step 1272, the second blade is removed from a blade holder. After step 1272, processing proceeds to a step 1274 where miniature processing is performed on the second blade. After step 1274, processing proceeds to a step 10-1276 where the second slide is scanned and imaged according to techniques similar to those discussed further elsewhere in the present invention. It is noted that in various embodiments the scanning and imaging techniques may include pre-scan focusing steps and/or that use dynamic focusing techniques, such as a real-time focusing technique. After step 1276, processing proceeds to a step 1278 where the second slide is deposited back into the slide holder. After step 1278, processing is completed with respect to the second blade. In accordance with an embodiment of the system described in the present invention involving slide storage, the steps of flowchart 1250 with respect to the first slide may be performed by the slide storage device in parallel with the steps of flowchart 1270 with respect to the second slide in order to to reduce cycle time. For example, steps 1272, 1274, and 1278 for the second slide (e.g. pick-up, thumbnail processing, and deposit) can overlap step 1256 for the first slide (e.g., scan/imaging the first slide) and vice versa. versa, in such a way that the times for picking, processing of thumbnails and depositing are eliminated from the cycle time. The cycle time is THE
W is therefore determined only by the scan time of a blade according to an embodiment of the system described in the present invention.
Figure 26 is a schematic illustration showing a blade storage device 1300 in accordance with another embodiment of the system described in the present invention. Blade storage device 1300 may include blade holder configured as a carousel 1310, blade handle 1320, storage 1330 and an XY platen.
1340. Carousel 1310 may include one or more positions 1312, 1312', 1312" 10 where blades, such as blade 1301, may be placed before and/or after being imaged by an imaging device 1350 which may have similar features and functionality to those discussed elsewhere in the present invention. Positions 1312, 1312', 1312" are shown as an arrangement of wedges (e.g. 8 wedges) and 15 as discussed further elsewhere in In the present invention, carousel 1310 may be of such height that multiple blade positions extend below each of the top level wedge positions 1312, 1312', 1312" that are shown. Blade handle 1320 may include an arm 1322 which acts as a pick-up head and may include 20 mechanical and/or vacuum devices for withdrawing a blade. Arm 1322 on blade handle 1320 may move between positions 1322a-d to ---- -move the blades within the carousel 1 310, the 1330 store and the XY platinum
1340. Storer 1330 may include multiple store positions 1332, 1334. A store position 1332 may be designated as a return store position 1332 in which slides that are returned from the imaging device 1350 through the XY stage 1340 can be positioned before being moved, at W
0
" ""'
blade handle 1320, back to carousel 1310. Another storage position 1334 may be designated as a camera storage position 1334 where a blade that is to be sent to the imaging device 1350 may first have an image in miniature
5 captured from the slide according to the techniques discussed elsewhere in the present invention.
After a thumbnail image of the slide is captured at camera store position 1334, the slide can be moved to a position 1342 on the XY stage 1340 which transports the slide "to the imaging device 1350 for scanning and imaging
10 in accordance with the techniques discussed elsewhere in the present invention.
Figure 27 is a schematic illustration showing another view of blade storage device 1300. Components of blade storage device 1300 may be capable of operating with multiple motions and multiple degrees of freedom of the blade.
15 movement.
For example, carousel 1310 may be rotatable in one direction 1311 and may include multiple blade positions 1312a-d at multiple height positions in each rotational position to accommodate multiple blades (shown as Blades 1, 2, 3 and 4). In one embodiment, multiple blade positions 1312a-d at each of the wedge positions
20 1312, 1312', 1312" may include positions for 40 blades, e.g. equidistantly positioned within the height of carousel 1310 that
""" may" measure, in one realization, 30.48 centimeters (12 inches). Furthermore, carousel 1310 may also include a user tray 1314 that has one or more blade positions 1314a,b in which a user can insert a
25 slide to be imaged in addition to other slides on the carousel 1310. Interaction of a slide in the user tray 1314, for example, lifting a cover from the user tray 1314 and/or inserting the slide into one of the positions 1314a,b from the 1314 user tray, can act
R à to trigger an override mode where a blade from user tray 1314 is processed instead of the next blade from carousel wedge positions 1310.
Arm 1322 of blade handle 1320 is shown to have at least three degrees of freedom in motion. For example, the arm 1322 can rotate in a direction 1321a in order to engage each of the carousel 1310, the store 1330 and the XY stage 1340. Additionally, the arm 1322 can be adjustable in a direction 1321b that corresponds to different heights of the positions 1312a-d of carousel 1310. Additionally, arm 1322 may extend in direction 1321C in connection with loading and unloading carousel blades 1310, storage 1330 and XY stage
1340. In one embodiment, it is advantageous to minimize the arc distance that the arm 1322 rotates and/or minimize other distances traversed by the arm 1322 and/or Blade handle 1320 in order to minimize dead times of the blade storage device 1300 , as discussed further below. The movements of carousel 1310, blade handle 1320 and XY stage 1340 may be controlled, in various embodiments, by a control system similar to that discussed elsewhere in the present invention. It is also noted that, in one embodiment, the storage 1330 and the XY platinum 1340 may be of the same height.
Figures 28A-28J are schematic illustrations showing blade storage operations of the blade storage device of Figures 26 and 27 in accordance with an embodiment of the system described in the present invention. In accordance with one embodiment, the blade operations 25 discussed in the present invention minimize system downtime, i.e., times during blade transfer and take-out operations that do not overlap with blade scanning and imaging operations.
Downtimes can include, for example, a parking time
0' - ^_ ., and wherein an XY stage 1340 moves into a position to allow blade handle 1320 to withdraw the blade. Other dead time contributions include moving the blade to the 1330 stockpile return position and reloading the 1340 XY stage with a blade.
5 Figure 28A begins the illustrated sequence where a slide 2 is currently being scanned and imaged on imaging device 1350. Slides 1, 3 and 4 are waiting to be scanned and imaged on carousel 1310 and the blade handle 1320 is in position to have delivered blade 2 to the XY stage 1340.
Figure 28B shows that blade handle 1320 rotates and descends to load the next blade (blade 3) to be scanned and imaged, while blade 2 continues to be scanned and imaged. Figure 28C shows that the blade handle 1320 transports the blade 3 to the camera store position 1334 of the store 15 1330 so that a thumbnail image is obtained of the blade 3. Figure 28D shows that the blade handle 1320 is positioned to discharge blade 2 from the XY stage 1340 that is returning from the imaging device 1350 after the blade 2 scan is completed. Note that the timing as the XY 1340 plate moves into the position to be discharged is an example of slow timing. The time after an XY 1340 stage is in position to be unloaded with blade 2 waiting there to be unloaded and blade 3 waiting to be loaded onto the 1340 XY stage is an example of parking time. Figure 28E shows that blade 2 is transported by blade handle 25 1320 from XY stage 1340 to return position 1332 of store 1330. Blade handle 1320 then proceeds to the position to withdraw blade 3 from the release position. camera store 1334. Figure 28F shows that blade 3 is removed from the camera store position
P 0 1334 and discharged onto the XY stage 1340. Figure 28G shows that blade 3 is currently being swept while blade 2 is being withdrawn from the store return position 1332 by the blade handle
1320. Figure 28H shows that the blade 2 is returned to its position on the carousel 1310 by the blade handle 1320 which rotates and moves translationally to the appropriate position. Figure 281 shows that the blade handle 1320 moves translationally to the proper position to remove the blade 1 from the carousel 1310. Figure 28J shows that the blade handle 1320 transports and unloads the blade 1 in the storage position. 10 in which a thumbnail image of blade 1 is taken, while blade 3 is still currently being scanned. Additional interactions, similar to those discussed above in connection with the illustrated sequencing, can be performed with respect to any of the remaining blades (e.g., blade 4) on carousel 1310 and/or for any 15 user-entered blades in the user tray 1314 to initiate the bypass mode operation discussed in the present invention.
Additionally, in accordance with the system described in the present invention, an illumination system can be used in connection with microscopy performances that are applicable to various techniques and attributes of the system described in the present invention. It is known that microscopes can commonly use Kohler illumination for brightfield microscopy. The primary attributes of Kohler illumination are that the area and numerical aperture of the illumination are both controllable through adjustable iris such that the illumination can be adapted to machine a wide range of microscope objectives with numerical aperture, field of view and magnification. variables. Kohler lighting offers desirable results, but requires multiple components that take up a significant amount of space. Accordingly, various embodiments of the system described herein are W -
The invention further provides attributes and techniques for advantageous lighting in microscopy applications that avoid certain disadvantages of known Köhler lighting systems while maintaining the advantages of Köhler lighting.
Figure 29 is a schematic illustration showing a lighting system 1400 for illuminating a blade 1401 using a light emitting diode (LED) lighting assembly 1402 in accordance with an embodiment of the system described in the present invention. The LED lighting assembly 1402 may have attributes according to multiple embodiments as discussed further in the present invention. Light from LED lighting assembly 1402 is transmitted through a mirror 1404 and/or other appropriate optical components to a capacitor 1406. Capacitor 1406 may be a capacitor that has a suitable working distance (for example, at least 28 mm ) to accommodate any required working distance of an XY 1408 plate, as discussed further elsewhere in the present invention. In one embodiment, the capacitor may be the SG03.0701 capacitor manufactured by Motic which has a working distance of 28 mm. Condenser 1406 may include an adjustable iris diaphragm that controls the numerical aperture (cone angle) of light illuminating the specimen on slide 1402. Slide 1401 may be arranged on XY stage 1408 under a microscope objective 1410. LED illumination light ' 1402 may be used in connection with scanning and imaging the specimen on slide 1401, including, for example, operations with respect to movement of an XY stage, slide storage, and/or dynamic focusing, from in accordance with the attributes and techniques of the system described in the present invention. The 1402 LED lighting assembly may include a 1420 LED, AND
C such as a bright white LED, a 1422 lens that can be used as q
II ~ q « ·r-- S e 67 à
a collector element and an adjustable Iris field diaphragm 1424 that can control the area of illumination on the blade 1401. The LED emission surface
1420 may be imaged by lens 1422 in an entrance pupil 1406a of condenser 1406. Entrance pupil 1406a may be
5 co-located with a NA 1406b adjustment diaphragm of the 1406 condenser. The 1422 lens can be chosen to collect a large fraction of the light output from the 1420 LED and also focus the image of the LED 1420 on the NA 1406b adjustment diaphragm of the 1406 condenser with the appropriate magnification so that the 1402 LED image fills the NA adjustment diaphragm opening
10 1406b of capacitor 1406. Capacitor 1406 can be used to focus LED light
1420 on blade 1401 with the NA 1406b adjustment diaphragm.
The illumination area on the 1401 blade can be controlled by the 1424 field diaphragm mounted on the 1402 LED lighting assembly. The field diaphragm and/or
The spacing between capacitor 1406 and field diaphragm 1424 can be adjusted to image the light from LED 1420 on blade piano 1401 so that field diaphragm 1424 can control the area of blade 1401 that is illuminated.
Since an image sensor acquires frames while the
20 Y platinum contains a moving blade, the 1420 LED can be pulsed (eg producing strobe motion) to allow very high brightness over a short time.
For example, for a Y stage moving at about 13 mm/second, to maintain a blur of no more than 0.5 pixel (0.250 micron/pixel), the 1420 LED can be pulsed while on.
25 for 10 microseconds.
The LED light pulse may be driven by a main clock closed to the oscillator lens resonant frequency in accordance with the techniques and focusing system discussed further elsewhere in the present invention.
© 0 — ·
Figure 30 is a schematic illustration showing a more detailed side view of an embodiment for an LED lighting assembly 1402' in accordance with the system described in the present invention and corresponding to the attributes described in the present invention with respect to the lighting assembly. 1402. A deployment and configuration of an LED 1430, a lens 1432, and a field diaphragm 1434 are shown with respect to and in connection with other adjustment and support components 1436. Figure 31 is a schematic illustration showing an exploded view. of a specific implementation of a 10 LED lighting assembly 1402" according to an embodiment of the system described in the present invention that has attributes and functions similar to those discussed with respect to the LED lighting assembly 1402. An adapter 1451, assembly 1452 , clamp 1453 and mount 1454 can be used to securely mount and locate a 1455 LED in the 1402" LED Light Assembly of mode 15 to be securely positioned with respect to a 1462 lens. Screw and washer components 1456-1461 may additionally be used to secure and mount the 1402" LED lighting assembly. In various embodiments, the 1455 LED can be a PhlatLight CM-360 series white LED, Luminus, this is a bright white LED that has an optical output of 4,500 lumens and a long lifespan of 70,000 hours and/or a suitable LED made by Luxeon. The 1462 lens can be an MG 9P6mm lens, 12mm OD (outside diameter). A tube lens component 1463, adapter 1464, retaining ring, and battery tube lens component 1467 can be used to position and mount the lens 1462 with respect to the 25 field adjustable diaphragm component 1465. adjustable field 1465 may be a Ring Activated Iris Diaphragm, part number SM1D12D by Thor Labs. The 1466 Stack Tube Lens can be a P3LG Stack Tube Lens by Thor Labs. The 1463 tube lens can be a
W+0 P50D or P5LG tube lens by Thor Labs. Other washer 1468 and screw 1469 components may be used, where appropriate, to additionally mount and secure 1402" LED lighting assembly elements.
In accordance with the system described in the present invention, devices and techniques are provided for high-speed blade scanning for digital pathology applications in accordance with various embodiments of the system described herein. In one embodiment, a slide holder for a pathology microscope may include: (i) a tray in the form of a disc and (ii) a plurality of recesses formed in the tray wherein each recess is adapted to receive a slide and the recesses are arranged circumferentially in the tray. The tray may include a central spindle hole and two locking holes wherein the locking holes are adapted to collect in a drive adapted to rotate at high speed around an axis normal to the tray. The recesses 15 may be recesses machined at different angular positions on the tray. The recesses may have semi-circular protrusions to touch the blade but not to constrict the blade too much thereby allowing the blade to be substantially effortless. The recesses may also have a cutout that allows a finger holder for placing and extracting the blade from the recess by an operator. In various embodiments, the blade holder, and the operation thereof, may be used in connection with the features and techniques discussed elsewhere in this document for an imaging system. Figure 32 is a schematic illustration showing a high speed blade scanner 1500 in accordance with an embodiment of the system described herein which may be used in connection with digital pathology imaging. A blade holder 1510 may include a tray 1512 with recesses 1514a,b..n arranged in positions
,,. .
The 70 a and angular angles of an annular or circumferential ring 1515 in the tray 1512, and the recesses 1514a-n each can be sized to hold a blade
1501. Tray 1512 is illustrated as a circular disc and can be manufactured to hold a desired number of blades. For example, to hold 165 slides, tray 1512 can measure approximately 32.02 cm (13 inches) in diameter. Note that other configurations of blades and tray size and shape may be used, as appropriate, in connection with the system described herein, and the orientation and configuration of recesses 1514a-n e may be modified accordingly. A blade 10 may be placed in each recess 1514a-n of tray 1512, so that blade placement 1501 in recess 1514a, and tray 1512 may be placed in high speed blade scanner 1500. Tray 1512 may include a center spindle hole 1516C and two locking holes 1516a and 1516b that can engage with a driver that rotates blade holder 1510 at high speed around geometry axis 1518 in rotational direction 1519. Tray 1512 can be placed in a low-profile drawer, shown representationally at 1502, which can retract tray 1512 on device 1500.
Figure 33 is a schematic illustration showing a recess 20 1520 in a high speed blade scanner tray in more detail according to an embodiment of the system described herein. Recess 1520 can be any of recesses 1514a-n. Recess 1520 may include a plurality of semicircular protuberances, such that three protuberances 1522a-c, to touch blade 1501 but not to constrict blade 1501 too much, thereby allowing blade 1501 to be substantially effortless. A cutout 1523 allows a finger holder to place and extract blade 1501 from recess 1520 by an operator.
Centripetal accelerations, shown schematically by arrows 1521,
produced by the blade holder 1510/tray 1512 as it rotates around the axis 1518 may apply a small holding force to the blade 1501 to hold the blade 1501 in place while imaging takes place.
The containment force can be designed to be
5 at least 0.1 g initially by rotating tray 1512 at rates greater than 100 rpm to register blade 1501 against semicircular protuberances 1522a-c.
Once the blade 1501 is registered, the rotation rate can be reduced consistent with system imaging rates as discussed elsewhere in this document.
At lower rates, even a small holding force would stabilize blade 1501 against the bulges 1522a-c.
Referring again to Figure 32, an imaging system microscope 1530, such as is discussed in detail elsewhere herein, may be disposed above tray 1512 to image the areas of the circumference ring 1515 where the slides are placed.
The 1530 imaging system may include a 1532 high NA microscope objective, e.g. 0.75 NA with a large working distance, a 1534 intermediate lens, and the 1536 CCD or CMOS 2D array image sensor placed in the distance. suitable for magnifying objects on the 1501 blade to the 1536 image sensor. The 1536 image sensor can have a high frame rate, so that it is greater than 100 frames/sec. for example, the 1536 image sensor can be part of a Dalsa Falcon 1.4M1OO camera that operates at 100 frames/sec. or equivalent.
The 1530 imaging system can be rigidly mounted on a 2-axis motorized drive that can be constructed of components such as DC motors or stepper motors, ball or lead screws and/or linear guides.
Since the geometry axis, radial geometry axis 1531a can move the system
.
Imaging F 1530, or at least a component thereof, radially through small movements, for example, 1 mm steps at a resolution of 10 microns to image one or more rings on the turntable 1512 below. The other axis, the focus geometry axis 1531b, moves in small motions of 5 to 10 microns with a resolution of 0.1 micron. The focus geometry axis can be built to perform movements at high speed, for example, performing a small movement in a few milliseconds. The objective movements of the 1534 microscope can be controlled by a control system and can be used in connection with dynamic focusing techniques such as are discussed elsewhere in this document.
A lighting system 1540 can be placed below the turntable 1502 and includes a light source 1542 such that a high-brightness white LED, one or more optical path components 15 like a mirror 1544, and a capacitor 1546 , similar to the lighting components discussed elsewhere in this document. In one embodiment, the condenser and imaging paths of the microscope can be connected and move as a rigid body, such direction 1541 of movement of the illumination system 1540 is in the same direction as the radial direction 1531a of the generating system. 1530. In the focus direction 1531b, the imaging path may be disengaged from the capacitor path, so that one or more components of the imaging system 1530 may include independent movements in the focus direction 1531b to perform 25 high-speed focus movements. Figure 34 is a schematic illustration showing an imaging trajectory starting at a first radial position with respect to blade 1501 to image a specimen 1501 Na u - —- P.
W blade 1501 in recess 1520. Recess 1520 with blade 1501 rotates with blade holder 1510 in rotational direction 1524. Images may be captured to frames (e.g., frames 1525) in accordance with the image capture techniques discussed in other parts of this document. As shown, images are captured for a row of frames (eg, 1525 frames) for each slide in the 1510 slide holder as the 1512 tray rotates under the 1530 imaging system. After one complete revolution of the 1512 tray, the Radial position of the 1530 imaging system is increased to capture 10 images for another row of frames for each blade. Each frame is acquired at a high rate and temporarily freezes the scene below. Brightfield lighting can be bright enough to allow for such short exposures. These exposures can be in a time frame of a few 10 to a few hundred microseconds. The process is continued until an image of the entire area of interest is generated for each slide in the slide holder 1510. In connection with this realization, processing the collected images into a mosaic image of an area of interest requires proper arrangement and/or image tagging to correctly correlate the multiple rows of frames between the 20 multi-slides that are rotated in tray 1512. Appropriate imaging processing techniques can be used to tag images in order to correlate captured images with the blade as long as the arcuate motion of the image block collection can be addressed by known splicing software and can be transformed to views that a pathologist would understand when observing using a standard microscope.
As an example, with a 33.528 centimeters (13.2 inches) diameter disc-shaped tray rotating at 6 rpm, an objective
- 74
W 6 "
20x microscope P of NA=0.75 produces a field of view of about 1 mm square. This arcuate field of view is traversed in about 10 milliseconds. For a tissue section within an active area of 15 mm square and assuming 25% overlap between fields, 20 fields 5 would need to be incremented along the radial axis. If the frame transfer is short enough not to limit the acquisition time, 20 full rotations would be enough to image 16 blades on the disk. This would occur at 6 rpm in 200 seconds or a transfer rate of 1 blade every 12.5 seconds.
Figures 35A and 35B are schematic illustrations showing an alternative arrangement of blades in a rotating blade holder according to another embodiment of the system described in the present invention. Figure 35A shows a tray 1512' with recesses 1514' configured such that the longest dimension of the blade 1501 is oriented along the radius of the disc-shaped tray 1512' which rotates in the direction 1519'. In this configuration, more blades (eg 30 blades) can fit in the 1512' tray. Figure 35B is a schematic view showing an imaging path for blade 1501 in a recess 1520' which is configured as noted above. In the illustrated embodiment, the blade 1501 is held in the recess 1520' according to the centripetal forces shown in the direction 1521' and the protuberances 1522a'-c'. The direction of rotation 1524' through which image processing is performed is shown for image collection for frames 1525' for specimen 1501'. The radial position of the imaging system 1530 is incremented in increments of with respect to the length of the blades to capture the images for successive rows of frames for each blade. In one example, for an active area of 15mm x 15mm and assuming a 25% overlap between fields. Twenty fields would need to be incremented along the geometry axis
Go to radial. Again, 20 rotations at 6 rpm would provide full imaging in 200 seconds, but with a more efficient sweep providing the blades orientation and thus the throughput will increase to one blade every 6.67 seconds.
Figure 36 is a schematic illustration showing an imaging system 1550 in accordance with an embodiment of the system described in the present invention that includes an objective 1552 arranged to examine a specimen 155 on a slide 1551." In one embodiment, focus positions can be predetermined by slower rotation of the disk prior to image acquisition. Budgeting as much as 20 seconds per blade for autofocus would make a total scan time under 30 seconds per blade - an order of magnitude faster than the current state of the art systems. As a tray 1560, on which the blade 1551 is arranged, rotates in the 1561 direction, the objective 1552 may undergo 15 minute movements in the 1562 direction to be positioned in the best focus as determined in accordance with the system described in the present invention.
Distinct autofocus values would not need to be set for each field of view 1553, but apply for larger distinct zones 1554 on blade 1551, e.g. 3 x 3 fields of view or subframes due to the 20 greater spatial frequencies of tissue thickness or blade bend. The autofocus values would be interpolated by applying the best focus as the blade moves under the camera on its arc path. Alternatively, a dynamic focusing technique, such as real-time focusing techniques described elsewhere in the present invention, may be advantageously employed in connection with the high-speed scanning systems provided in the present invention. Note that the times to acquire focus points (eg, 120 focus points per second) allow for the use of real-time focusing in conjunction with the high-speed rotational scanning techniques discussed above. It is further noted that it is well within the field of control systems to control a rotating disk at speeds within 1 part in 10,000, allowing open-loop sampling of each image without relying on rotational feedback from the disk. Generally, a low resolution thumbnail image is produced of the blade. This can be accomplished by setting a low resolution camera over an angled position of the disk so as not to interfere with the high resolution microscope described at this point. For extremely high volume applications, the disk shape lends itself to manipulation. The 300 mm (-12") disks of semiconductor chip handling robots can be used to move disks from a stockpile to the high-speed scanning device. Furthermore, most technologies position the slide under the microscope objective through linear stages in one step and repeated motion. These movements dominate the image acquisition times. The system described in the present invention that uses a rotary motion is efficient and highly repeatable. Image acquisition and autofocus times are an order of magnitude less than current state of the art products.
Most systems also require locking mechanisms or spring retainers to hold the blade in place during the stop and advance motions of the stage. The system described in the present invention does not require a detent mechanism where rotational motion creates a centripetal acceleration that pushes the blade to a predetermined location in a recess cut in the disc. This makes building the blade holder simpler and more reliable. Additionally, blade retainers can bend or stretch the blade which complicates autofocus processes and are advantageously avoided according to the system described in
W 0 the present invention. Current systems have peak speeds of 2 to 3 minutes for an active area of 15 mm per blade. The systems and methods provided in the present invention allow the same area to be scanned under 30 seconds, 5 for the example outlined above. Many pathology labs aim to scan 100 slides to 200 slides per day. With these high image acquisition rates an operator can work through a daily inventory of slides within an hour including the added steps of loading and unloading discs, scanning barcode 10, pre-focus. This allows for faster time to result and improved savings for the lab. Figure 37 is a flowchart 1600 showing high speed blade scanning using a rotating tray in accordance with an embodiment of the system described in the present invention. In a step 1602, the 15 blades are located in the recesses of the rotating tray. After step 1602, processing proceeds to a step 1604 in which the rotating tray is moved to a slide scan position with respect to an imaging and scanning system. After step 1604, processing proceeds to a step 1606 where rotation of the turntable is initiated.
20 As discussed above, rotation of the rotating tray causes centripetal forces acting on the blades to hold the blades in a desired imaging position. After step 1606, processing proceeds to a step 1608 where the imaging system captures images, in accordance with the systems and techniques described in the present invention and including dynamic focusing techniques, for a row of frames for each slide. in a circumferential ring of the rotating tray. After step 1608, processing proceeds to a test step 1610 where it is determined whether a desired area of interest on each slide in the tray is
W#78'W'—0 roundabout was scanned and imaged. If not, then processing proceeds to a step 1612 where the imaging system and/or certain components of the meso are moved one increment in a radial direction of the rotating tray. After step 1612, processing 5 proceeds back to step 1608. If, in test step 1610, it is determined that the area of interest on each slide has been scanned and imaged, processing proceeds to a step 1614 where one or more mosaic images are created corresponding to the imaged areas of interest for each sheet. After step 1614, processing is completed. Additionally in accordance with the system described in the present invention, an optical duplication technique and device can be provided and used in connection with the imaging system attributes described in the present invention. In one embodiment, the system described in the present invention can sample a resolution element produced by a 20x 0.75 NA Plan Apo objective. This resolution element is about 0.5 micron at a wavelength of 500 nm. To further sample this resolution element, the tube lens in front of the imaging sensor can be changed. An approximate calculation 20 to compute the focal length of the tube lens, given the objective lens. _ , (f tube lens = focal length of tube lens in front of image sensor) is: pix_ sensor = pixel size! on the CCD image sensor or
CMOS 25 pix _ object= pixel size in object or fabric f tube lens = pix_ object/pix_ sensor * 9 mm . To obtain an object pixel size of 0.25 micron for the Dalsa Falcon 4M30/60 (7.4 micron sensor pixel), the focal length of the
+
B tube lens should be about 266mm. For a pixel size on the object of 0.125 micron, the focal length of the tube lens should be about 532 mm. It may be desirable to switch between these two object pixel sizes and this may be accomplished by mounting two or more tube lenses to a plate that transfers in front of the image sensor. Given the different path lengths associated with each new focal length, bent mirrors will also need to be added to bend the path to a fixed image sensor position.
Figure 38 is a schematic illustration showing an optical doubling imaging system 1700 in accordance with an embodiment of the system described herein. Optical duplication imaging system 1700 may include an image sensor 1710 of a camera 1711 and a microscope objective 1720 as described elsewhere in this document. It is noted that other components in connection with the system and techniques discussed in this document, such as a real-time focusing system, may also be used with the illustrated optical duplication imaging system 1700. To achieve two or more object pixel sizes, a plurality of tube lenses, for example, a first tube lens 1740 and a second tube lens 1750, may be provided in connection with the system described herein. A stage 1730 can carry the first tube lens 1740 and the second tube lens 1750, respectively, in front of the imaging sensor. In one embodiment, the stage 1730 can be a linearly activated stage that moves in a direction 1731, although it is noted that other types of stages and movement thereof 25 can be used in connection with the system described herein. A mirror assembly 1752 is shown in relation to the second tube lens 1750 which may include one or more dual mirrors to adjust the light path from the second tube lens 1750 to the image sensor 1710.
Figures 39A and 39B are schematic illustrations of the optical doubling imaging system 1700 showing the transport of the first tube lens 1740 and the second tube lens 1750 in front of the image sensor 1710 in accordance with a realization of the system described in this document. Figure 39A shows a light path 1741 for the first tube lens 1740 positioned in front of the image sensor 1710 on the stage.
1730. Figure 39B shows a light path 1751 for the second tube lens 1750 after being transported in front of the sensory image 1710 via the stage 1730. As illustrated, the light path 1751 has been augmented using 10 or more mirrors. of the mirror array 1752. In the two figures, it is seen that the optical doubling imaging system 1700 may include other appropriate optical and structural components 1760 such as those discussed in detail elsewhere in this document. Various embodiments discussed in this document may be combined with each other in appropriate combinations in connection with the system described in this document. Additionally, in some cases, the order of steps in the flowchart, flowcharts, and/or described flow processing may be modified, where appropriate. In addition, various aspects of the system described in this document may be implemented using software, hardware, a combination of software and hardware, and/or other computer-deployed modules or devices that have the described features and perform the described functions. Software deployments of the system described in this document may include executable code that is stored on a computer-readable storage medium and executed by one or more processors. The computer readable storage medium may include a computer hard disk, ROM, RAM, flash memory, portable computer storage medium such as a CD-ROM, a DVD-ROM, a drive and/or other drives with, for example, ,
.-. - W
J a universal bus (USB) interface, and/or any other suitable tangible storage medium or computer memory in which executable code can be stored and executed by a processor. The system described in this document can be used in connection with any appropriate operating system. Other embodiments of the invention will be apparent to those skilled in the art from a consideration of the specification or practical invention disclosed herein. It is an object that the specification and examples be considered as exemplary only, with the scope and true spirit of the invention being indicated by the following claims.

权利要求:
Claims (15)
[1]
1. DEVICES TO OBTAIN A FOCUSED IMAGE OF A SPECIMEN, characterized in that it comprises: an objective lens (120, 1410, 1552, 1720) arranged for examination 5 of the specimen (1501'); a slow focusing plate (140) coupled to the objective lens (120, 1410, 1552, 1720), wherein the slow focusing plate (140) controls movement of the objective lens (120, 1410, 1552, 1720); an oscillator focus stage (150) including an oscillator lens, wherein the oscillator focus stage (150) moves the oscillator lens; a focus sensor (160) that provides focus information in accordance with light transmitted through the oscillator lens; at least one electrical component that uses the focus information to determine a metric and a first focus position of the objective lens (120, 1410, 1552, 1720) according to the metric, wherein at least one electrical component sends information of position the slow focusing stage (140) for moving the objective lens (120, 1410, 1552, 1720) to the first focusing position; and 20 an image sensor (110, 1536, 1710) that captures an image (110') of the specimen (1501') after the objective lens (120, 1410, 1552, 1720) is moved to the first focus position.
[2]
2. DEVICE, according to claim 1, characterized in that it further comprises: 25 an XY movement plate (130, 1340, 1408), in which the specimen (1501') is arranged on the XY movement plate (130 , 1340, 1408), and wherein at least one electrical component controls the movement of the XY motion plate (130, 1340, 1408).
THE
W 2 u .
:and
[3]
3. DEVICE, according to claim 2, characterized in that the movement of the XY movement plate (130, 1340, 1408) is blocked in phase with the movement of the oscillator lens.
[4]
4. DEVICE, according to claim 1, 5 characterized in that the oscillator focus plate (150) includes a flexed assembly driven by a moving coil! which moves the oscillator lens - . in a translational motion. .
[5]
5. Disposal according to claim 1, characterized in that the oscillator lens is moved at a resonant frequency that is at least 60 Hz, and wherein at least one electrical component uses the focus information to perform at least 60 focus calculations per second.
[6]
6. DEVICE, according to claim 1, characterized in that the focus sensor (160) and the oscillator focus plate (150) are adjusted to operate bidirectionally, and the focus sensor (160) produces the focus information on both the top and bottom portion of a sinusoidal waveform of the oscillator lens motion at the resonant frequency.
[7]
7. DEVICE according to claim 1, 20 characterized in that the metric includes at least one of: contrast information, sharpness information, and chroma information.
[8]
8. DlSPOSlTlVO according to claim 1, characterized in that the focus information includes information for a plurality of zones (402') of a focus window (402) that is used during a focus scan of the specimen (1501').
[9]
9. METHOD TO OBTAIN A FOCUSED IMAGE OF A SPECIMEN, characterized in that it comprises: controlling the movement of an objective lens (120,· 1410, 1552,
q
The 3 0 .
q 1720) arranged for specimen examination (1501'); controlling the motion of an oscillator lens; providing focus information according to the light transmitted through the oscillator lens; 5 use the focus information to determine a metric and determine a first objective lens focus position (120, 1410, 1552, 1720) according to the metric: and send position information that is used to move the objective lens ( 120, 1410, 1552, 1720) for the first focus position.
[10]
10 10. METHOD, according to claim 9, characterized by the fact that the first focus position is determined as a best focus position (120'), and the method also comprises: capturing an image (110') of the specimen (1501') after the objective lens (120, 1410, 1552, 1720) is moved to the best focus position 15 (120').
[11]
11. METHOD, according to claim 9, characterized in that it further comprises: controlling the movement of an XY movement plate (130, 1340, 1408) on which the specimen (1501') is arranged.
[12]
20 12. METHOD, according to claim 9, characterized in that the oscillator lens is moved at a resonant frequency -which-is- of at least 60 Hz, and in which at least 60 focus calculations are performed per second.
[13]
13. METHOD, according to claim 9, characterized by the fact that the metric includes at least one of: sharpness information, contrast information and chroma information.
[14]
14. METHOD TO OBTAIN AN IMAGE OF A SPECIMEN, characterized by the fact that it comprises:
B - 4 establishing a nominal focus plane (101'); position the specimen (1501') at a start position that has associated x and y coordinates; and performing first processing in a single pass on said specimen (1501'), said first processing includes: determining, for each of a plurality of points, a focus position using an oscillator lens; and acquiring, for each of said plurality of points, a frame according to said focus position.
10
[15]
15. COMPUTER READable MEDIUM, characterized in that it comprises a code stored therein to obtain a focused image (110') of a specimen (1501'), as defined in one of claims 9 to 14.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112012009241A2|2020-09-01|&#39;&#39;device for obtaining a focused image of a specimen, method, computer readable medium, device for a microscope stage, device for slide cache, device for slide scanning and method for scanning the slide&#39;&#39;

---

CA2844994C|2016-10-11|Imaging systems, cassettes, and methods of using the same

---

AU2013205438B2|2014-10-16|Imaging system and techniques

---

AU2015201450B2|2016-11-03|Imaging systems, cassettes, and methods of using the same

同族专利:

---

公开号 | 公开日

---

CN104020553B|2017-06-16|

---

SG10201403755RA|2014-10-30|

---

WO2011049608A3|2011-11-17|

---

KR101423896B1|2014-07-28|

---

HK1201332A1|2015-08-28|

---

JP2014098907A|2014-05-29|

---

US9575301B2|2017-02-21|

---

EP2993510A1|2016-03-09|

---

CN102687061A|2012-09-19|

---

US20150153552A1|2015-06-04|

---

IN2012DN02336A|2015-08-21|

---

EP2993478A2|2016-03-09|

---

EP2993510B1|2020-06-24|

---

WO2011049608A2|2011-04-28|

---

US20120312957A1|2012-12-13|

---

KR20130092635A|2013-08-20|

---

KR101420030B1|2014-07-15|

---

CA2842661A1|2011-04-28|

---

JP5819390B2|2015-11-24|

---

AU2010308550A1|2012-04-05|

---

JP5627698B2|2014-11-19|

---

US20150153555A1|2015-06-04|

---

CA2842723C|2016-02-16|

---

CN104020554B|2017-08-08|

---

JP2013508746A|2013-03-07|

---

HK1201333A1|2015-08-28|

---

CN104020553A|2014-09-03|

---

IL241746D0|2015-11-30|

---

CN104020556A|2014-09-03|

---

IL241747D0|2015-11-30|

---

IL241745D0|2015-11-30|

---

US9036255B2|2015-05-19|

---

CA2842661C|2016-02-23|

---

KR20120056877A|2012-06-04|

---

EP2491452A2|2012-08-29|

---

JP2014112229A|2014-06-19|

---

HK1176683A1|2013-08-02|

---

EP2993478A3|2016-05-25|

---

CN104020554A|2014-09-03|

---

AU2010308550B2|2013-05-23|

---

IL241745A|2016-09-29|

---

KR101423897B1|2014-07-28|

---

JP5819389B2|2015-11-24|

---

KR101424414B1|2014-08-01|

---

CA2842723A1|2011-04-28|

---

IL241747A|2016-09-29|

---

CA2776527A1|2011-04-28|

---

HK1201334A1|2015-08-28|

---

JP5902144B2|2016-04-13|

---

CA2842721A1|2011-04-28|

---

IL218822A|2016-02-29|

---

CA2842721C|2016-03-29|

---

SG187479A1|2013-02-28|

---

CN102687061B|2014-12-10|

---

US20150153556A1|2015-06-04|

---

SG187478A1|2013-02-28|

---

KR20130092636A|2013-08-20|

---

EP2491452A4|2015-03-11|

---

CN104020556B|2016-08-17|

---

KR20130092637A|2013-08-20|

---

CA2776527C|2014-08-05|

---

EP2990855A1|2016-03-02|

---

JP2014098908A|2014-05-29|

---

IL241746A|2016-09-29|

---

IL218822D0|2012-06-28|

---

US10061107B2|2018-08-28|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

GB190603092A|1906-02-08|1906-11-01|Hector Lebrun|Table for Microscopes.|

---

US2788485A|1953-05-22|1957-04-09|Chauncey L Parsons|Translator drive|

---

US2788455A|1955-11-04|1957-04-09|Ingraham E Co|Self-starting synchronous motor|

---

FR81726E|1962-05-23|1963-11-02|Centre Nat Rech Scient|Interferential measurement method and its applications|

---

US3233949A|1962-08-01|1966-02-08|Stocker & Yale Inc|Sliding mechanisms|

---

DE2218156A1|1971-06-18|1972-12-21|Bausch & Lomb|Slide finger for microscope|

---

JPS5235212A|1975-09-16|1977-03-17|Daiichi Nenryo Kogyo|Method of making ironworks dust harmless|

---

JPS6034740B2|1976-02-27|1985-08-10|Nippon Kogaku Kk|

---

JPS52143848A|1976-05-26|1977-11-30|Hitachi Ltd|Thin plate external appearance inspector|

---

GB2034913A|1978-11-21|1980-06-11|Coal Industry Patents Ltd|Microscope stage|

---

US4433235A|1980-10-09|1984-02-21|Hitachi, Ltd.|Focusing position detecting device in optical magnifying and observing apparatus|

---

US4453807A|1981-06-17|1984-06-12|Smithkline Beckman Corp|System for handling slides|

---

JPS61239780A|1985-04-16|1986-10-25|Matsushita Electric Ind Co Ltd|Auto-focus device|

---

JPS6235212A|1985-08-09|1987-02-16|Nippon Kogaku Kk <Nikon>|Observing device|

---

US5202931A|1987-10-06|1993-04-13|Cell Analysis Systems, Inc.|Methods and apparatus for the quantitation of nuclear protein|

---

JPH07122694B2|1986-10-16|1995-12-25|オリンパス光学工業株式会社|Illumination device for microscope|

---

US4844617A|1988-01-20|1989-07-04|Tencor Instruments|Confocal measuring microscope with automatic focusing|

---

US5075079A|1990-05-21|1991-12-24|Technicon Instruments Corporation|Slide analysis system|

---

US5367401A|1990-11-23|1994-11-22|Perceptive Scientific Instruments, Inc.|Microscope slide rotary stage|

---

US5162641A|1991-02-19|1992-11-10|Phoenix Laser Systems, Inc.|System and method for detecting, correcting and measuring depth movement of target tissue in a laser surgical system|

---

FR2673478B1|1991-03-01|1994-02-04|Data Fit International|AUTOMATIC SLIDE DIGITIZATION STATION.|

---

JPH054122U|1991-04-30|1993-01-22|オリンパス光学工業株式会社|Transmission optical microscope|

---

JPH054122A|1991-06-28|1993-01-14|Brother Ind Ltd|Method and device for wire-cut electric discharge machining|

---

GB2258109B|1991-07-25|1995-05-17|Sony Broadcast & Communication|Autofocus systems|

---

JP2768065B2|1991-08-06|1998-06-25|住友金属工業株式会社|Speed control method of die set cutting machine|

---

DE4131360A1|1991-09-20|1993-03-25|Jenoptik Jena Gmbh|MICROSCOPE HANDLING SYSTEM|

---

US5428690A|1991-09-23|1995-06-27|Becton Dickinson And Company|Method and apparatus for automated assay of biological specimens|

---

JP2565793Y2|1991-10-29|1998-03-18|オリンパス光学工業株式会社|Moving stage|

---

US5912699A|1992-02-18|1999-06-15|Neopath, Inc.|Method and apparatus for rapid capture of focused microscopic images|

---

JPH0694987A|1992-09-03|1994-04-08|Nikon Corp|Focus detector|

---

GB2273994A|1992-12-18|1994-07-06|Morphometrix Inc|Process microscopy system|

---

US5784162A|1993-08-18|1998-07-21|Applied Spectral Imaging Ltd.|Spectral bio-imaging methods for biological research, medical diagnostics and therapy|

---

US5432871A|1993-08-04|1995-07-11|Universal Systems & Technology, Inc.|Systems and methods for interactive image data acquisition and compression|

---

US6165734A|1995-12-12|2000-12-26|Applied Spectral Imaging Ltd.|In-situ method of analyzing cells|

---

US5532873A|1993-09-08|1996-07-02|Dixon; Arthur E.|Scanning beam laser microscope with wide range of magnification|

---

JPH0883753A|1994-09-13|1996-03-26|Nikon Corp|Focal point detecting method|

---

US5846749A|1994-10-12|1998-12-08|The Regents Of The University Of California|Quantitative measurement of tissue protein identified by immunohistochemistry and standardized protein determination|

---

JP3541260B2|1995-01-20|2004-07-07|株式会社ニコン|Cross carriage|

---

US5589938A|1995-07-10|1996-12-31|Zygo Corporation|Method and apparatus for optical interferometric measurements with reduced sensitivity to vibration|

---

WO1997012975A1|1995-10-02|1997-04-10|The Government Of The United States Of America, Represented By The Secretary, Department Of Health And Human Services|An epithelial protein and dna thereof for use in early cancer detection|

---

US5981279A|1995-11-09|1999-11-09|Allegheny University Of The Health Sciences|Compositions and methods to regulate calmodulin gene expression, and uses thereof for influencing cell growth and differentiation|

---

US5768033A|1996-06-14|1998-06-16|Brock; Dennis|Microscope assembly comprising a supported and movable specimen wheel and fine adjustment means|

---

US6396941B1|1996-08-23|2002-05-28|Bacus Research Laboratories, Inc.|Method and apparatus for internet, intranet, and local viewing of virtual microscope slides|

---

JPH10197802A|1997-01-10|1998-07-31|Nikon Corp|Stage|

---

US6665008B1|1997-07-15|2003-12-16|Silverbrook Research Pty Ltd|Artcard for the control of the operation of a camera device|

---

US5982535A|1997-08-07|1999-11-09|Marine Biological Laboratory|Centrifuge microscope capable of realizing polarized light observation|

---

GB9721991D0|1997-10-18|1997-12-17|Secr Defence|Optical systems|

---

DE19746575A1|1997-10-22|1999-04-29|Zeiss Carl Fa|Optical image recording device and method for its use|

---

WO1999022318A1|1997-10-27|1999-05-06|Massachusetts Institute Of Technology|Image search and retrieval system|

---

US6183981B1|1998-03-05|2001-02-06|Board Of Regents, The University Of Texas System|Diagnostic assay for late-onset Alzheimer's disease|

---

US6185030B1|1998-03-20|2001-02-06|James W. Overbeck|Wide field of view and high speed scanning microscopy|

---

US6562299B1|1998-09-18|2003-05-13|Cytyc Corporation|Method and apparatus for preparing cytological specimens|

---

JP2000241713A|1999-02-16|2000-09-08|Leica Microsystems Wetzler Gmbh|Holder for sample carrier|

---

DK1145066T3|1998-12-21|2005-07-04|Evotec Ag|Positioning the measurement volume in a scanning microscopic method|

---

US6118582A|1999-07-12|2000-09-12|Immuno Concepts, Inc.|Slide holder|

---

US7006674B1|1999-10-29|2006-02-28|Cytyc Corporation|Apparatus and methods for verifying the location of areas of interest within a sample in an imaging system|

---

US7505614B1|2000-04-03|2009-03-17|Carl Zeiss Microimaging Ais, Inc.|Remote interpretation of medical images|

---

CN2418493Y|2000-04-26|2001-02-07|王雷鸣|Multi-working-position microscope article-carrying table|

---

JP3794670B2|2000-04-28|2006-07-05|株式会社日立国際電気|Microscope autofocus method and apparatus|

---

US7738688B2|2000-05-03|2010-06-15|Aperio Technologies, Inc.|System and method for viewing virtual slides|

---

US7518652B2|2000-05-03|2009-04-14|Aperio Technologies, Inc.|Method and apparatus for pre-focus in a linear array based slide scanner|

---

US6711283B1|2000-05-03|2004-03-23|Aperio Technologies, Inc.|Fully automatic rapid microscope slide scanner|

---

US7027628B1|2000-11-14|2006-04-11|The United States Of America As Represented By The Department Of Health And Human Services|Automated microscopic image acquisition, compositing, and display|

---

US6993169B2|2001-01-11|2006-01-31|Trestle Corporation|System and method for finding regions of interest for microscopic digital montage imaging|

---

US7155049B2|2001-01-11|2006-12-26|Trestle Acquisition Corp.|System for creating microscopic digital montage images|

---

TW528881B|2001-02-05|2003-04-21|Hitachi Int Electric Inc|Position measuring apparatus|

---

US6816606B2|2001-02-21|2004-11-09|Interscope Technologies, Inc.|Method for maintaining high-quality focus during high-throughput, microscopic digital montage imaging|

---

US7012575B2|2001-07-27|2006-03-14|Samsung Electronics Co., Ltd.|Deceleration module integrated with rolling device for automatically extending/retracting antenna and automatic antenna extending system using the same|

---

US6697509B2|2001-10-04|2004-02-24|Chromavision Medical Systems, Inc.|Method and apparatus for scoring the uptake of markers in cells|

---

JP2003255234A|2002-03-05|2003-09-10|Lasertec Corp|Microscope and imaging method|

---

JP2003270545A|2002-03-14|2003-09-25|Olympus Optical Co Ltd|Optical scanning stage|

---

US7012757B2|2002-05-08|2006-03-14|Be Here Corporation|Method for designing a lens system and resulting apparatus|

---

US6788457B2|2002-05-30|2004-09-07|George Mauro|Microscope stage|

---

AU2003245499A1|2002-06-14|2003-12-31|Chromavision Medical Systems, Inc.|Automated slide staining apparatus|

---

US7116440B2|2003-02-28|2006-10-03|Aperio Technologies, Inc.|Image processing and analysis framework|

---

US7345814B2|2003-09-29|2008-03-18|Olympus Corporation|Microscope system and microscope focus maintaining device for the same|

---

WO2005050563A2|2003-11-17|2005-06-02|Aureon Biosciences Corporation|Pathological tissue mapping|

---

JP2005202092A|2004-01-15|2005-07-28|Hitachi Kokusai Electric Inc|Focusing point detecting method and optical microscope using the same|

---

JP2005266584A|2004-03-19|2005-09-29|Olympus Corp|Automatic focusing method and its device|

---

EP1580586B1|2004-03-25|2008-06-11|Olympus Corporation|Scanning confocal microscope|

---

WO2005093388A1|2004-03-26|2005-10-06|Infectio Recherche Inc.|Removable microfluidic flow cell|

---

US7066004B1|2004-09-02|2006-06-27|Sandia Corporation|Inertial measurement unit using rotatable MEMS sensors|

---

US20060050376A1|2004-09-02|2006-03-09|Houston Edward S|Robotic microscopy apparatus for high throughput observation of multicellular organisms|

---

JP2005024567A|2004-09-07|2005-01-27|Hitachi Kokusai Electric Inc|Location finding apparatus|

---

US7113625B2|2004-10-01|2006-09-26|U.S. Pathology Labs, Inc.|System and method for image analysis of slides|

---

JP2006281553A|2005-03-31|2006-10-19|Seiko Epson Corp|Printer, method of detecting stick slip, program, and printing system|

---

US7417213B2|2005-06-22|2008-08-26|Tripath Imaging, Inc.|Apparatus and method for rapid microscopic image focusing having a movable objective|

---

US7508583B2|2005-09-14|2009-03-24|Cytyc Corporation|Configurable cytological imaging system|

---

JP2007086559A|2005-09-26|2007-04-05|Pentax Corp|Camera|

---

JP4708143B2|2005-09-30|2011-06-22|シスメックス株式会社|Automatic microscope and analyzer equipped with the same|

---

JP4354446B2|2005-10-13|2009-10-28|株式会社東海ヒット|Microscope stage and microscope observation unit|

---

JP4680052B2|2005-12-22|2011-05-11|シスメックス株式会社|Specimen imaging apparatus and specimen analyzer including the same|

---

EP2036003B1|2006-06-30|2017-05-03|Leica Biosystems Imaging, Inc.|Method for storing and retrieving large images via dicom|

---

US8010555B2|2006-06-30|2011-08-30|Aperio Technologies, Inc.|System and method for managing images over a network|

---

US8067245B2|2006-07-24|2011-11-29|Medica Corporation|Automated microscope for blood cell analysis|

---

US7742209B2|2006-11-01|2010-06-22|Inphase Technologies, Inc.|Monocular holographic data storage system architecture|

---

US7697831B1|2007-02-20|2010-04-13|Siimpel Corporation|Auto-focus with lens vibration|

---

WO2008118886A1|2007-03-23|2008-10-02|Bioimagene, Inc.|Digital microscope slide scanning system and methods|

---

US8565498B2|2007-04-27|2013-10-22|Leica Biosystems Imaging, Inc.|Second opinion network|

---

US7576307B2|2007-04-30|2009-08-18|General Electric Company|Microscope with dual image sensors for rapid autofocusing|

---

US8878923B2|2007-08-23|2014-11-04|General Electric Company|System and method for enhanced predictive autofocusing|

---

JP5389016B2|2007-05-04|2014-01-15|アペリオ・テクノロジーズ・インコーポレイテッド|System and method for quality assurance in pathology|

---

US7869022B2|2007-07-18|2011-01-11|Asml Netherlands B.V.|Inspection method and apparatus lithographic apparatus, lithographic processing cell, device manufacturing method and distance measuring system|

---

US8330768B2|2007-07-27|2012-12-11|Sharp Laboratories Of America, Inc.|Apparatus and method for rendering high dynamic range images for standard dynamic range display|

---

US8629382B2|2007-09-03|2014-01-14|Nikon Corporation|Auto focus apparatus for detecting a focal point with a setting section for shifting an irradiation position outside an observation field of an imaging section|

---

JP2009098437A|2007-10-17|2009-05-07|Nikon Corp|Focus adjustment device and microscope device|

---

CN201141938Y|2007-12-24|2008-10-29|颜睿|Rotary microscope carrier|

---

US20110190153A1|2008-06-09|2011-08-04|Nils Adey|System and method for hybridization slide processing|

---

CN100595624C|2008-10-24|2010-03-24|陈百捷|Moving table for microexamination|

---

KR101423896B1|2009-10-19|2014-07-28|벤타나 메디컬 시스템즈, 인코포레이티드|Device and method for slide caching|JP5739351B2|2009-03-11|2015-06-24|サクラ ファインテック ユー．エス．エー．， インコーポレイテッド|Automatic focusing method and automatic focusing device|

---

KR101423896B1|2009-10-19|2014-07-28|벤타나 메디컬 시스템즈, 인코포레이티드|Device and method for slide caching|

---

JP2011180442A|2010-03-02|2011-09-15|Sony Corp|Device, method and program for acquisition of sample image|

---

JP5577885B2|2010-06-28|2014-08-27|ソニー株式会社|Microscope and focusing method|

---

US20130169788A1|2010-10-29|2013-07-04|Canon Kabushiki Kaisha|Microscope, image acquisition apparatus, and image acquisition system|

---

BR112014016072B1|2011-12-29|2021-01-12|Sight Diagnostics Ltd.|method and system for detecting a plasmodium infection in a blood sample|

---

US9522396B2|2010-12-29|2016-12-20|S.D. Sight Diagnostics Ltd.|Apparatus and method for automatic detection of pathogens|

---

DE102011121928B4|2011-08-01|2015-03-05|Physik InstrumenteGmbh & Co. Kg|Arrangement for operating a dynamic nanofocusing system|

---

EP2753966A1|2011-09-09|2014-07-16|Ventana Medical Systems, Inc.|Focus and imaging system and techniques using error signal|

---

GB2542911B|2011-10-14|2017-07-26|Solentim Ltd|Method of and apparatus for analysis of a sample of biological tissue cells|

---

US10591501B2|2011-10-31|2020-03-17|Torsten Matthias|Automatic structure determination|

---

AU2013214345B2|2012-02-01|2016-09-15|Ventana Medical Systems, Inc.|System for detecting genes in tissue samples|

---

US9841590B2|2012-05-02|2017-12-12|Leica Biosystems Imaging, Inc.|Real-time focusing in line scan imaging|

---

JP6019798B2|2012-06-22|2016-11-02|ソニー株式会社|Information processing apparatus, information processing system, and information processing method|

---

JP6022285B2|2012-09-28|2016-11-09|シスメックス株式会社|Specimen storage device, specimen storage method, and specimen inspection system|

---

JP5923026B2|2012-10-31|2016-05-24|浜松ホトニクス株式会社|Image acquisition apparatus and image acquisition method|

---

DE102012223533A1|2012-12-18|2014-06-18|Carl Zeiss Microscopy Gmbh|Digital microscope system|

---

US9001422B2|2013-02-28|2015-04-07|Ventana Medical Systems, Inc.|Slide handling system and techniques|

---

JP2014178357A|2013-03-13|2014-09-25|Sony Corp|Digital microscope device, imaging method of the same and program|

---

TWI625551B|2013-03-15|2018-06-01|傲思丹度科技公司|3d light field displays and methods with improved viewing angle depth and resolution|

---

EP2973397B1|2013-03-15|2017-08-02|Ventana Medical Systems, Inc.|Tissue object-based machine learning system for automated scoring of digital whole slides|

---

DE102013103971A1|2013-04-19|2014-11-06|Sensovation Ag|Method for generating an overall picture of an object composed of several partial images|

---

WO2014188405A1|2013-05-23|2014-11-27|Parasight Ltd.|Method and system for imaging a cell sample|

---

CN104251811A|2013-06-28|2014-12-31|西门子医疗保健诊断公司|Digital microscope and image identification method|

---

CN103353670A|2013-08-01|2013-10-16|陈松|Full-automatic electrically-controlled microscope sample bench|

---

US10831013B2|2013-08-26|2020-11-10|S.D. Sight Diagnostics Ltd.|Digital microscopy systems, methods and computer program products|

---

JP6173154B2|2013-10-01|2017-08-02|オリンパス株式会社|Microscope system|

---

US20150177158A1|2013-12-13|2015-06-25|General Electric Company|Operational performance assessment of additive manufacturing|

---

US10007102B2|2013-12-23|2018-06-26|Sakura Finetek U.S.A., Inc.|Microscope with slide clamping assembly|

---

CA2947926C|2014-06-27|2020-03-10|Ventana Medical Systems, Inc.|Automated specimen processing systems and methods of detecting specimen-bearing microscope slides|

---

CN107076964B|2014-08-06|2020-01-03|赛洛米克斯股份有限公司|Image-based laser auto-focusing system|

---

CN107003505B|2014-08-13|2021-03-05|丹尼尔·萨默·加鲁|Line scanning, sample scanning, multi-mode confocal microscope|

---

US10482595B2|2014-08-27|2019-11-19|S.D. Sight Diagnostics Ltd.|System and method for calculating focus variation for a digital microscope|

---

EP3201628B1|2014-10-02|2020-11-18|Ventana Medical Systems, Inc.|Polymers and conjugates comprising the same|

---

CA2965564A1|2014-11-10|2016-05-19|Ventana Medical Systems, Inc.|Classifying nuclei in histology images|

---

CN112697792A|2014-12-26|2021-04-23|希森美康株式会社|Cell imaging device, cell imaging method, and sample cell|

---

JP2018503906A|2014-12-30|2018-02-08|ベンタナ メディカル システムズ， インコーポレイテッド|System and method for co-expression analysis in immunoscore calculation|

---

WO2016120440A1|2015-01-29|2016-08-04|Ventana Medical Systems, Inc.|Dot detection, color classification of dots and counting of color classified dots|

---

WO2016120442A1|2015-01-30|2016-08-04|Ventana Medical Systems, Inc.|Foreground segmentation and nucleus ranking for scoring dual ish images|

---

WO2016120441A2|2015-01-30|2016-08-04|Ventana Medical Systems, Inc.|Quality metrics for automatic evaluation of dual ish images|

---

CA2977073A1|2015-02-23|2016-09-01|Li-Cor, Inc.|Fluorescence biopsy specimen imager and methods|

---

JP6862345B2|2015-03-12|2021-04-21|コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ．|Lighting unit for digital pathology scanning|

---

AU2016236323A1|2015-03-20|2017-08-10|Ventana Medical Systems, Inc.|System and method for image segmentation|

---

JP6134348B2|2015-03-31|2017-05-24|シスメックス株式会社|Cell imaging device and cell imaging method|

---

EP3295173A1|2015-05-10|2018-03-21|Ventana Medical Systems, Inc.|Compositions and methods for simultaneous inactivation of alkaline phosphatase and peroxidase enzymes during automated multiplex tissue staining assays|

---

WO2016189065A1|2015-05-26|2016-12-01|Ventana Medical Systems, Inc.|Method and system for assessing stain quality for in-situ hybridization and immunohistochemistry|

---

EP3869184A1|2015-06-26|2021-08-25|Li-Cor, Inc.|Fluorescence biopsy specimen imager and methods|

---

AU2016316846B2|2015-08-28|2021-07-08|Ventana Medical Systems, Inc.|Protein proximity assay in formalin fixed paffafin embedded tissue using caged haptens|

---

CN108474934B|2015-09-17|2022-01-18|思迪赛特诊断有限公司|Method and apparatus for detecting entities in a body sample|

---

CA3000053A1|2015-09-24|2017-03-30|Leica Biosystems Imaging, Inc.|Real-time focusing in line scan imaging|

---

KR20170043821A|2015-10-14|2017-04-24|삼성전자주식회사|Apparatus for auto focus control, and method for manufacturing semiconductor device|

---

CA3002148A1|2015-10-16|2017-04-20|Mikroscan Technologies, Inc.|Systems, media, methods, and apparatus for enhanced digital microscopy|

---

CN105301794B|2015-10-30|2017-08-25|中国科学院遗传与发育生物学研究所|Super-resolution imaging device for fast moving objects|

---

DE102015119590A1|2015-11-12|2017-05-18|Carsten Jung|lighting arrangement|

---

WO2017132276A1|2016-01-26|2017-08-03|Ventana Medical Systems, Inc.|Predictive diagnostic workflow for tumors usnig automated dissection, next generation sequencing, and automated slide stainers|

---

EP3426684A1|2016-03-08|2019-01-16|Ventana Medical Systems, Inc.|Multiplexed immunohistochemistry using recombinant antibodies with epitope tags|

---

CN107205113B|2016-03-18|2020-10-16|松下知识产权经营株式会社|Image generation device, image generation method, and computer-readable storage medium|

---

JP6742783B2|2016-04-01|2020-08-19|株式会社ミツトヨ|Imaging system and imaging method|

---

EP3446098A1|2016-04-21|2019-02-27|Li-Cor, Inc.|Multimodality multi-axis 3-d imaging|

---

WO2017195208A1|2016-05-11|2017-11-16|S.D. Sight Diagnostics Ltd|Performing optical measurements on a sample|

---

US10768406B2|2016-06-20|2020-09-08|Scanogen Inc.|Imaging system|

---

US10278586B2|2016-06-23|2019-05-07|Li-Cor, Inc.|Complementary color flashing for multichannel image presentation|

---

WO2018002015A1|2016-06-28|2018-01-04|Ventana Medical Systems, Inc.|New colors for chromogenic ihc and ish staining with multi-dye quinone methide and tyramide conjugates|

---

WO2018002016A1|2016-06-28|2018-01-04|Ventana Medical Systems, Inc.|Application of click chemistry for signal amplification in ihc and ish assays|

---

US20180045937A1|2016-08-10|2018-02-15|Zeta Instruments, Inc.|Automated 3-d measurement|

---

JP2019533149A|2016-09-23|2019-11-14|ヴェンタナ メディカル システムズ， インク．|Method and system for scoring extracellular matrix biomarkers in tumor samples|

---

EP3545488A1|2016-11-23|2019-10-02|Li-Cor, Inc.|Motion-adaptive interactive imaging method|

---

US10613308B2|2016-11-30|2020-04-07|Yuefeng YIN|Method and microscope for measuring and calculating heights on curved surface of microscope slide|

---

WO2019120635A1|2017-12-18|2019-06-27|Ventana Medical Systems, Inc.|Peptide nucleic acid conjugates|

---

CN110121364A|2016-12-19|2019-08-13|文塔纳医疗系统公司|Peptide nucleic acid conjugate|

---

JP2020514701A|2016-12-19|2020-05-21|ヴェンタナ メディカル システムズ， インク．|Method and system for quantitative immunohistochemistry|

---

EP3559672B1|2016-12-21|2021-10-13|Ventana Medical Systems, Inc.|Methods and systems for reagent delivery|

---

CN110088804A|2016-12-22|2019-08-02|文塔纳医疗系统公司|It is scored based on the computer of primary colors and immunohistochemistry image|

---

US10663711B2|2017-01-04|2020-05-26|Corista, LLC|Virtual slide stagemethod for viewing whole slide images|

---

US10783627B2|2017-03-03|2020-09-22|Case Western Reserve University|Predicting cancer recurrence using local co-occurrence of cell morphology |

---

EP3605185A4|2017-03-30|2020-03-25|FUJIFILM Corporation|Observation device, observation control method, and observation control program|

---

JP6889009B2|2017-04-13|2021-06-18|浜松ホトニクス株式会社|Image acquisition system and image acquisition method|

---

CN110494891A|2017-04-14|2019-11-22|文塔纳医疗系统公司|It local registration for splicing based on piecemeal and global places|

---

WO2018200261A1|2017-04-25|2018-11-01|Li-Cor, Inc.|Top-down and rotational side view biopsy specimen imager and methods|

---

EP3396430A1|2017-04-27|2018-10-31|Euroimmun Medizinische Labordiagnostika AG|Optical scanning arrangement and method|

---

EP3631410A2|2017-05-26|2020-04-08|Ventana Medical Systems, Inc.|Non-contact, on-slide fluid mixing|

---

JP2020525772A|2017-06-28|2020-08-27|ヴェンタナ メディカル システムズ， インク．|System level calibration|

---

WO2019020556A1|2017-07-24|2019-01-31|Ventana Medical Systems, Inc.|Methods and systems for evaluation of immune cell infiltrate in tumor samples|

---

EP3662484A1|2017-08-04|2020-06-10|Ventana Medical Systems, Inc.|Automatic assay assessment and normalization for image processing|

---

CN111602172A|2017-08-04|2020-08-28|文塔纳医疗系统公司|Color unmixing with scatter correction|

---

WO2019068038A1|2017-09-29|2019-04-04|Leica Biosystems Imaging, Inc.|Real-time autofocus focusing algorithm|

---

CN111183387A|2017-10-04|2020-05-19|徕卡生物系统成像股份有限公司|Opposed edge system for scanning and processing glass slides|

---

EP3467563A1|2017-10-06|2019-04-10|Siemens Healthcare GmbH|Microscope device|

---

WO2019101658A1|2017-11-21|2019-05-31|Ventana Medical Systems, Inc.|Contactless mixing using modulated air jets|

---

EP3625570A4|2017-11-30|2021-06-09|Leica Biosystems Imaging, Inc.|Safety light curtain to disable carousel rotation|

---

JP2021506013A|2017-12-05|2021-02-18|ベンタナ メディカル システムズ， インコーポレイテッド|How to Calculate Tumor Spatial Heterogeneity and Intermarker Heterogeneity|

---

CN111448569A|2017-12-06|2020-07-24|文塔纳医疗系统公司|Method for storing and retrieving digital pathology analysis results|

---

JP2021506022A|2017-12-07|2021-02-18|ベンタナ メディカル システムズ， インコーポレイテッド|Deep learning systems and methods for solidarity cell and region classification in bioimaging|

---

WO2019121564A2|2017-12-24|2019-06-27|Ventana Medical Systems, Inc.|Computational pathology approach for retrospective analysis of tissue-based companion diagnostic driven clinical trial studies|

---

WO2019149817A1|2018-01-31|2019-08-08|Ventana Medical Systems, Inc.|Methods and systems for evaluation of immune cell infiltrate in stage iii colorectal cancer|

---

CN108827920A|2018-03-21|2018-11-16|苏州国科医疗科技发展有限公司|A kind of low Poison bleaching co-focusing imaging method and system|

---

WO2019197509A1|2018-04-13|2019-10-17|Ventana Medical Systems, Inc.|Systems for cell shape estimation|

---

US10706328B2|2018-05-07|2020-07-07|Google Llc|Focus-weighted, machine learning disease classifier error prediction for microscope slide images|

---

EP3794548A1|2018-05-15|2021-03-24|Ventana Medical Systems, Inc.|Quantitation of signal in stain aggregates|

---

WO2019224153A1|2018-05-21|2019-11-28|Genentech, Inc.|Her2 heterogeneity as a biomarker in cancer|

---

EP3815040A4|2018-05-21|2022-03-02|Corista LLC|Multi-sample whole slide image processing via multi-resolution registration|

---

EP3824288A1|2018-07-17|2021-05-26|Ventana Medical Systems, Inc.|Materials and methods for detecting fusion proteins|

---

EP3830742A1|2018-07-27|2021-06-09|Ventana Medical Systems, Inc.|Systems for automated in situ hybridization analysis|

---

EP3853214A1|2018-09-20|2021-07-28|Ventana Medical Systems, Inc.|Coumarin-based crosslinking reagents|

---

EP3861556A1|2018-10-01|2021-08-11|Ventana Medical Systems, Inc.|Methods and systems for predicting response to pd-1 axis directed therapeutics|

---

EP3867868A1|2018-10-15|2021-08-25|Ventana Medical Systems, Inc.|Image enhancement to enable improved nuclei detection and segmentation|

---

EP3867803A1|2018-10-15|2021-08-25|Ventana Medical Systems, Inc.|Systems and methods for cell classification|

---

EP3884259A1|2018-11-20|2021-09-29|Ventana Medical Systems, Inc.|Methods and systems for preparing and analyzing cellular samples for morphological characteristics and biomarker expression|

---

RU2703106C1|2019-01-29|2019-10-15|Общество с ограниченной ответственностью "Медицинские компьютерные системы "|Method of selecting the bundled and scanning modes of the adapted multifunctional scanning microscope|

---

EP3921648A1|2019-02-05|2021-12-15|Ventana Medical Systems, Inc.|Methods and systems for evaluation of immune cell infiltrate in stage iv colorectal cancer|

---

KR102237696B1|2019-04-29|2021-04-08|계명대학교 산학협력단|A artificial intelligence computer aided diagnosis system for automated analysis of cervical cancer cell image and its control method|

---

CN110132967B|2019-05-14|2021-10-08|南京凯泽瑞兹光电科技有限公司|Digital pathological section scanner|

---

WO2020229578A1|2019-05-14|2020-11-19|Ventana Medical Systems, Inc.|System including a biological sample treatment chamber|

---

KR102236513B1|2019-05-24|2021-04-06|주식회사 필로시스|Scanning device and driving method of the same|

---

WO2021025675A1|2019-08-05|2021-02-11|Hewlett-Packard Development Company, L.P.|Apparatus and method to determining nozzle health in 3d printing|

---

CN110441898A|2019-08-21|2019-11-12|苏州精濑光电有限公司|A kind of microscope and its control method|

---

WO2021037872A1|2019-08-28|2021-03-04|Ventana Medical Systems, Inc.|Label-free assessment of biomarker expression with vibrational spectroscopy|

---

WO2021037875A1|2019-08-28|2021-03-04|Ventana Medical Systems, Inc.|Systems and methods for assessing specimen fixation duration and quality using vibrational spectroscopy|

---

WO2021037869A1|2019-08-28|2021-03-04|Ventana Medical Systems, Inc.|Assessing antigen retrieval and target retrieval progression quantitation with vibrational spectroscopy|

---

WO2021108584A1|2019-11-27|2021-06-03|Fluidigm Canada Inc.|Automated and high throughput imaging mass cytometry|

---

KR102358718B1|2019-12-19|2022-02-04|서울과학기술대학교 산학협력단|Modulized sample stage|

---

WO2021224293A1|2020-05-07|2021-11-11|Ventana Medical Systems, Inc.|Histochemical systems and methods for evaluating egfr and egfr ligand expression in tumor samples|

---

RU2755569C1|2020-07-21|2021-09-17|Общество с ограниченной ответственностью "Медицинские компьютерные системы "|Method for collective training of automatic microscopy combines analysing biomaterial preparations|

---

WO2022041197A1|2020-08-31|2022-03-03|深圳迈瑞生物医疗电子股份有限公司|Sample image photographing method and sample image photographing apparatus|

---

CN112051250B|2020-09-09|2021-11-23|南京诺源医疗器械有限公司|Medical fluorescence imaging image light supplement adjusting system and adjusting method|

法律状态:
2020-09-08| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2021-10-13| B07A| Application suspended after technical examination (opinion) [chapter 7.1 patent gazette]|

---

2021-11-23| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US25299509P| true| 2009-10-19|2009-10-19|

---

US61/252,995|2009-10-19|

---

US25622809P| true| 2009-10-29|2009-10-29|

---

US61/256,228|2009-10-29|

---

US26125109P| true| 2009-11-13|2009-11-13|

---

US61/261,251|2009-11-13|

---

US29923110P| true| 2010-01-28|2010-01-28|

---

US61/299,231|2010-01-28|

---

US36734110P| true| 2010-07-23|2010-07-23|

---

US61/367,341|2010-07-23|

---

PCT/US2010/002772|WO2011049608A2|2009-10-19|2010-10-18|Imaging system and techniques|

---